HN Gopher Feed (2017-10-09) - page 1 of 10
Facebook Quietly Enters StarCraft War for AI Bots
174 points by sus_007https://www.wired.com/story/facebook-quietly-enters-starcraft-wa...
Merem - 3 hours ago
"The contest Facebook entered, like most AI research in the area,
used an older version of StarCraft, which is considered equally
difficult for software to master."I'm surprised this one hasn't
been called out yet but Starcraft: Brood War and Starcraft II are
completely different games, not just different versions.And apart
from what was already mentioned regarding information on the scene,
this link is pretty useful:
http://sscaitournament.com/index.php?action=tutorialAn older post
on this topic: http://www.teamliquid.net/blogs/485544-intro-to-
scbw-ai-deve...Facebook: facebook.com/groups/bwapiAs for SSCAIT in
bfung - 3 hours ago
Hmm, are they truly completely different games?From a high level,
it seems like the elements for an AI to learn are mostly the
same: Resource gathering and optimization, map/location battle
optimizations, building/technology decisions...It's not like
Starcraft: verison X is different than Starcraft: version Y is
different the way Starcraft and Hearthstone are different games.
XR0CSWV3h3kZWg - 3 hours ago
Honestly from an AI perspective SC:BW and SC2 are probably more
similar than from a human's perspective. 12+ unit selection,
multi building selection & intelligent worker rallies aren't
huge improvements for AI.
haeffin - 2 hours ago
SC2 ticks way faster than SC1 though, which you don't notice
much as a human, but you might change your strategy when
designing the ai components.
XR0CSWV3h3kZWg - 1 hours ago
Also I'd imagine the amount of time you spend making sure
that units will path efficiently is way less!
FlorianRappl - 2 hours ago
I agree with the sentiment that for a human the difference is
much larger than for a machine, however, taking into
consideration that a huge amount of the AI in SCBW needs to be
on the micro (i.e., in-battle) execution which is - for the
most part - handled by the in-game AI in SC2 I feel both games
are different enough on AI development. Some basics are
definitely shared, but beyond a certain level the two games go
in different directions.One thing to add is the openBW project
(http://www.openbw.com). It is truly amazing: The guys there
managed to reverse engineer the full SCBW engine (I thought
that's nearly impossible, as the whole flow depends a lot on
implementation details, some of the details are even considered
bugs, which turned out to be just the right implementation to
balance the game perfectly...).
david-cako - 1 hours ago
AI has a significant edge in micro and mechanics in BW that is
much less significant in SC2.
taw55 - 2 hours ago
I haven?t played either, but I know for a fact that they use
different pathfinding methods (grid vs navmesh, II has swarm
lazzlazzlazz - 2 hours ago
They are truly completely different games.
jasonlotito - 31 minutes ago
So which one isn't an RTS?
vm - 6 hours ago
Misleading title. FB seems to be taking a self-learning approach,
whereas hobbyists are defining bot rules themselves. Hobbyists are
winning at the moment with an approach that doesn't scale. Self-
learning has a better chance of dominating the field.Edit: HN mods
updated title to be less click-baity (thanks). Earlier it said
"FACEBOOK QUIETLY ENTERS STARCRAFT WAR FOR AI BOTS, AND LOSES". The
"and loses" at the end was misleading.
toxik - 6 hours ago
As someone who has implemented quite a few reinforcement learning
techniques and seen their limitations, I would be surprised if RL
could overcome handcrafting for SC any day soon.
AstralStorm - 6 hours ago
The main way the AI bots have problems is with timing. The
neutral networks used have no way of encoding time dependent
actions in a reasonable way. (As opposed to say a fuzzy
decision tree with explicit time input.) And if you try to
explicitly include it, curse of dimensionality strikes back
hard.Both absolute and relative timing have to be handled. And
relative since specific salient action...Plus the real reward
is very sparse. Say, crippling mineral production early may or
may not snowball. Likewise being a unit or two up...
eutectic - 3 hours ago
I suspect you might be able to do surprisingly well with just
a few simple features, e.g. what did I last see at each
position and how long ago was that, how many of each enemy
unit have I seen simultaneously and at what time, etc.As to
the sparsity of reward, I'm not sure this is such a big
problem. Once the AI learns that e.g. 'resources are good',
it can then learn how to optimize resource production. You
could even give the process a head start by learning a
function of time+various resources+assorted features to win
rate from human games to use as the reward function.
NikolaeVarius - 3 hours ago
"Resources are good" doesn't really mean anything.Yes
resources are good, but how do you know when to
expand?Judging from opponents movements, you can tell if
they're turtling, going for some cheese strat, or doing
some build where they may not be able to respond to a
aggressive expansion.Of course if you choose wrong, you
lost the game.
comicjk - 6 hours ago
What that tells me is that they haven't yet come up with the
right featurization - that is, the function that maps input
data into the actual neural network node values. The
appropriate featurization would include the time information
but reduce its dimensionality by hard-coding some basic
assumptions, of the kind that humans presumably make when
processing the same data.
yingxie3 - 5 hours ago
I think these guys (and most people using deep models) try
to avoid hand-crafted features as much as possible.
eutectic - 3 hours ago
Alphago used several hand-crafted features as of the
Nature paper, so DeepMind at least is not above a little
unpwn - 5 hours ago
Why do you say that? The dota2 bot open ai did earlier this
year seemed pretty convincing and similar...
dxhdr - 5 hours ago
Starcraft is a much larger, more complex, more freeform game
than Dota 2. It's like Go compared to chess.
cli - 5 hours ago
I disagree with this (I used to play Warcraft 3, and
currently play Dota 2), but that's beside the point. The
Dota 2 OpenAI is only set up for one mirror matchup
(impossible in real games) involving one hero on each side,
in one lane, and only for the first 10-ish minutes. This is
maybe 1% of a real Dota game.
kazagistar - 5 hours ago
I think you are both correct. Starcraft has a far larger
space of verbs at any given moment, and many of them can
impact each other, giving it one form of complexity,
while Dota2 clearly has a much more complex set of units
and abilities, leading to more possibilities total, even
if the number of possible actions moment to moment are
more limited. But yeah, the bot was a teeny little bit of
the game, impressive as it was.
sametmax - 5 hours ago
A lot of complexity in dota comes from the interactions
between 10 players. Make it 10 ai having to communicate
using chat. Make them pick and ban. Make all objects
available. And then you'll have real complexity.With
hundreds of unit x objects, jungling, roshan, cd and
pick + ban, you can actually get at the sc level of
Retric - 5 hours ago
It's 10 players vs ~200 bots.So, SC is still a much more
complex space. DoTA has non player bots, but they are
similar to SC buildings and follow very simple rules.
sametmax - 4 hours ago
You can't compare the complexity of a dota char with a
unit. Sc units are very basics, they don't have xp, don't
have a skill tree, don't have have 100 of possible
objects and their skills don't vary so much in context.
Retric - 3 hours ago
This matters far less than you might think. Go has
simpler peaces without movement and it's still much more
complex than chess.
eterm - 4 hours ago
If DOTA is anything like league I'm not sure I agree
completely. I think in league there's more 'future
prediction' needed, i.e. the current state is less
immediate than in star-craft. In star-craft you can quickly
see who is winning but in league there are things such as
pushing lanes to consider and knock-on effects from later
back timings (I know dota doesn't have backing but it has
couriers?).While that all can be extrapolated from current
state I think starcraft is much easier to go for immediate
gains by destroying more supply/resource value of units and
extrapolate from there.
cosarara97 - 3 hours ago
OpenAI was playing 1v1
SerLava - 3 hours ago
Starcraft strategy has a lot more weird nuance to it.I
noticed this building in this position at this time and I
haven't been attacked by X unit yet, so he's probably
doing strategy Y. I better skip some unrelated building I
was going to make, so I can have an extra unit Z in case
he's doing that strategy. Then I'll place the units at a
particular spot to try to trap him because that unit will
be vulnerable in this other spot so he's unlikely to move
through that spot.
dxhdr - 38 minutes ago
And then the mind games begin.
roenxi - 1 hours ago
A SC:BW bot will have the ability to be perfectly aware
of every unit in vision at all times.It wouldn't be a
suprise if some research team could put out a bot
achieving superhuman victories purely by out-microing an
opponent with minimal strategic choices.
dxhdr - 52 minutes ago
> It wouldn't be a suprise if some research team could
put out a bot achieving superhuman victories purely by
out-microing an opponent with minimal strategic
choices.The chess equivalent would be letting Deep Blue
take 10 years to evaluate each move; it's not a very
interesting system anymore since it isn't playing under
normal rules (~90 minutes per turn).Any "real" SC AI will
have limitations on input, say 300 actions per minute.
It'd be pretty interesting to see how few actions per
minute an AI could use to defeat the top human players.
SerLava - 12 minutes ago
>It wouldn't be a suprise if some research team could put
out a bot achieving superhuman victories purely by out-
microing an opponent with minimal strategic choices.Yeah
they did pretty much that. But the problem is it's a very
brute-force approach and violates some rules of the
game.They jam thousands of commands per second into the
game, and give each unit its own rudimentary AI. The
units basically just dance at maximum range, magically
dodge hits, etc.If they limit it to 600 actions per
minute (10 keystrokes hitting the keyboard every second -
still beyond the human mind but beyond human fingers) it
becomes a much harder AI problem.
romaniv - 5 hours ago
Didn't they restrict the game to a single hero and a subset
of all items to make it tractable for AI? And didn't people
find a way to break that bot?
thephyber - 2 hours ago
> And didn't people find a way to break that bot?According
to the player interviews and Reddit discussion threads, the
"break" you are talking about was more like being really
unpredictable therefore finding a play style that the AI
had never encountered.The players were flailing to find a
way to defend against the AI that is learning quicker over
time than they are.
cli - 5 hours ago
It was restricted to one very artificial game mode (1 vs 1
mid, Shadow Fiend mirror matchup) that is not
representative of real games, but is good for practicing
one aspect of Dota. Some items in this game mode are banned
in order to make it less safe for both sides, and so that
there is a better chance of one side dying before ten
minutes. People have beaten the bot by both "breaking" it
and by just outplaying it (but the latter very, very
smrtinsert - 5 hours ago
Factually correct title.
tree_of_item - 5 hours ago
Misleading? How? It's the title of the actual article, and it's
what happened. Just because they'll probably win in the future
because their approach is more sound, doesn't mean they didn't
vm - 4 hours ago
Makes it seem like they lost in an AI bot battle. But they are
in fact creating AI bots that compete against rule-based bots.
tree_of_item - 2 hours ago
They did lose in an AI bot battle. A rule based bot is still
rising-sky - 5 hours ago
Weak article but still surprised there's no mention of Open AI's
foray into Dota 2: https://blog.openai.com/dota-2/
crsv - 5 hours ago
I'm actually glad no one mentioned this because this event in and
of itself was overhyped publicity. The Open AI bot had hard
coded behaviors and was defeated handily within hours of being
available to the public. It's since been iterated upon and
there's still a handful of pros who are regularly beating it
WITHOUT exploiting the holes in the logic. This is effectively
just as weak as the article regarding facebook.
rising-sky - 3 hours ago
Interesting... I was not aware of this, thanks
hsrada - 4 hours ago
> and was defeated handily within hours of being available to
the public.The public exploited tricks to beat it. They did not
beat it 'handily'.Afterwards, the pros who do beat it only
manage to do so ~2-3 times for every 100 games played. I
believe they have been playing the same version that was shown
at The International and not an iterated version.
gdb - 3 hours ago
(I work at OpenAI.)Correct, we've been playing a number of
pros using the same bot played at TI. We do have a stronger
version which is just two days more of training (gets a 70%
win rate vs the one at TI), but haven't seen a need to test
it out. We'll likely do a blog post in upcoming weeks with
more stats and commentary from the pros; would be curious
what people would like to know!Incidentally, the various
exploits that people used are all similar to how we actually
develop the bot. We try to find areas of the strategy space
it hasn't explored, and then make a small tweak to encourage
it to explore that area. Lots of progress comes from removing
hardcoded restrictions, which are nice to get started. So the
fact there exist exploits wasn't surprising to us ? what
would be surprising would be exploits we couldn't fix.1v1 has
always been a proof-of-concept for us. The fun part is really
5v5, which is what we're working on now (and hiring for! ping
me if you're interested: firstname.lastname@example.org).
thephyber - 2 hours ago
> 1v1 has always been a proof-of-concept for us.I
understand this is a perfectly reasonable early test, but
there are so many complaints about "it was just a
restricted subset of the game and 1v1".This is like
complaining that Google doesn't release first-pass code
(with minimal unit tests and no stress testing) to their
production sites across the world. Everything that loops
starts with the first iteration.Also, keep up the good
work, OpenAI! And please remember Asimov's 3 rules.
huac - 5 hours ago
> A Microsoft research paper on machine learning this year said
that improving predictions of when a user will click on an ad by
just 0.1 percent would yield hundreds of millions of dollars in new
revenue.Do you really need a 'research paper on machine learning'
to understand that?
CamelCaseName - 5 hours ago
Does anyone have the link to this paper?I am curious if that
millions in revenue is for Microsoft (not surprising) or
advertisers (more interesting) - I would love to read through
their thought process either way.
icebraining - 5 hours ago
They need a research paper to show they're not pulling numbers
from their ass.
yeahbutbut - 5 hours ago
Do people really click on ads on purpose?
stagger87 - 2 hours ago
thephyber - 2 hours ago
Perhaps that's the trick -- if you make an ad look less like an
ad, it's 0.1% more likely to convert.And Google still hasn't
figured out how to best their search advertising network
business, so I assume that people click on ads, even if I'm not
in that demographic.You should really watch a child play a game
that has interstitial ads. It's quite obvious that they often
click on ads because they want to learn more (maybe not fully
convert, but intentionally click).
lfowles - 4 hours ago
Only slightly related, but does anyone remember the code puzzles
that FB had as a filter for the interview process? Specifically one
(jurassic island?) where you would connect to a shared server and
take control of a dinosaur on the island. Eventually when you
gathered enough energy to lay an egg, you could spawn another
connection to control that dino.I had no hope of having a
competitive score on the leaderboard, but I loved playing around
with that little world.Edit: Found more info on dinosaur island:
amenghra - 4 hours ago
Fun story, there was a pretty serious vulnerability in there
.Lessons learned: don't run CTFs "too close" to your real
Fiahil - 1 hours ago
Ah! I created a very similar game as a school project (a few
years ago) ! It was a 2nd year project, pardon the messy commits
infecto - 6 hours ago
Anyone have good resources on how to get into this? Would love to
learn more about self-learning in regards to starcraft.
make3 - 6 hours ago
the easier way : watch the course vids here : Berkeley CS294 Deep
Reinforcement Learning Fall 2017: http://www.youtube.com/playlist
?list=PLkFD6_40KJIznC9CDbVTjA...harder more complete way: read
this "Deep Reinforcement Learning: An Overview",
https://arxiv.org/abs/1701.07274 progressively implementing
subparts in Python and openai gym https://gym.openai.com/read-
etiam - 6 hours ago
oneweekwonder - 6 hours ago
blizzard have a github account with a couple of py libs for sc2
and other games.https://github.com/blizzard
pdimitar - 4 hours ago
IRC FreeNode, channel #BWAPI. Most of the authors of the bots
competing 24/7 in https://twitch.tv/sscait hang around in various
times of the day and night. They can provide you with plenty of
entry material.Few links:- https://sscaitournament.com/-
make3 - 6 hours ago
omg what a stupid title from wired. wired showing again that they
are a sensationalist piece of internet garbage
tree_of_item - 4 hours ago
They came in sixth. That's what happened. Exactly what is so
Jyaif - 6 hours ago
Yeah, I'm disappointed their articles keep popping up on HN.
dymk - 6 hours ago
If the title has an anti corporate slant, especially FB/Google,
it will continue to be upvoted on HN regardless of actual
fhood - 5 hours ago
Did somebody say anti-corporate slant?!?
m777z - 6 hours ago
6/28 is a very respectable finish, though I was surprised that the
top 3 finishers were just individuals, not even teams.
make3 - 6 hours ago
they're not learned ai, they're built in classical hard coded
logic perfected through the years, with lots of domain specific
knowledge from the users. that's a much easier way to produce
something that does okay than to innovate with a new ai scheme,
but importantly it also has no chance of ever beating a player
who's at the professional level of play, let alone a top one
yters - 4 hours ago
If AI weren't a religion, people would realize it is a joke by now.
pdimitar - 2 hours ago
One of the bots authors participating in SSCAIT
(https://twitch.tv/sscait) once sent me this
are supposed to be mini maps where your bot can train separate
aspects of the game on a much smaller scale.Extremely useful if
you're an aspiring bot author.
CyberDildonics - 4 hours ago
Quietly means paying for an article to be written in Wired now?
scj - 7 hours ago
Here's a link if you want to see CherryPi vs. Purplewave (two bots
Strilanc - 3 hours ago
That's surprisingly frustrating to watch. Protoss throwing away
its army of zealots with terrible positioning, zerg wasting time
attacking buildings instead of sealing the deal, probes walking
by enemy forces that ignore them as they mine minerals...Which is
not to say that there weren't impressive parts. I don't think a
human could stutter step that many dragoons that well. But
there's clearly a lot left to do in terms of tactics.
pdimitar - 2 hours ago
See my root-level comment in this thread:
9935c101ab17a66 - 3 hours ago
Thanks for the link, really cool to see the AIs at work.
horsecaptin - 6 hours ago
Is it fair to flag all articles that bait / mislead readers?
cgb223 - 6 hours ago
Where can I watch other Starcraft Bots play each other?Anyone have
links to the matches?
alexmorenodev - 3 hours ago
There is StarCraft Artificial Intelligence Tournament on youtube.
arcticfox - 5 hours ago
If anyone is interested in similar RTS AI play but for Age of
shagie - 6 hours ago
Twitch is where I would be digging around.
https://twitch.tv/sscait is one such with commentary.
ncds42 - 5 hours ago
Unless there are in-depth APM limitations on the AI, there's no
contest for AI vs. A human.
stale2002 - 4 hours ago
Umm, they have human vs AI challenges in starcraft and the AIs
are vastly outclassed by humans. No APM limits needed.AIs have a
long way to go to beat good human starcraft players.
thephyber - 2 hours ago
What you say is true right now, but in the long term, the gp is
correct unless we hit an AI asymptote which prevents AI from
approaching human StarCraft capabilities.Just from a pure
economy standpoint, any computer process has quite an advantage
from just optimizing action queues and keeping idle workers