The most interesting part of this announcement, I think, is the new capabilities and restrictions they disclosed for AlphaStar. It is now playing the latest version of the game, with all three races unrestricted, on multiple maps.
It also has much more realistic limits on its capabilities. Restricting the bot to one screen of information at a time is great. There's not much detail on the APM restrictions, but "designed with pro players" is a good sign. Hopefully they've designed some realistic restrictions more complex than just a cap on APM.
Of course the anonymous nature of the testing is interesting as well. Big contrast to OpenAI's public play test. I guess it will prevent people from learning to exploit the bot's weaknesses, as they won't know they are playing a bot at all. I hope they eventually do a public test without the anonymity so we can see how its strategies hold up under focused attack.
APM is an interesting question I think. Like what is the relative importance of APM to strategy?
If I have 20% more APM than you do I beat you even if I totally pick the wrong units and build order? Could a super smart bot win against pro's even if it's limited to 20% less APM?
So yeah it might be interesting if the bot gets superhuman to trail several versions with progressively lower APM caps to see how much you have to limit it before humans can beat it again.
APM is misleading because it can mean a lot of things. There are bots out there that will perfectly micro units to win battles that would typically trade out.
SC2 is really about attention management- every action you take, like moving a worker to construct a building that you'll have money for in the future, takes time and attention, and good players will take the minimal action needed to get something done and then move their attention elsewhere while they wait (almost like CPU pipelining). This is why players will do runbys/drop harass at the same time as a big fight. It forces the defending player to spend more attention than the attacker, or risk losing key workers/buildings. APM is just a rough representation of the ability to multitask.
From this perspective, AlphaStar's APM doesn't really matter. If it can instantly understand and juggle different scenarios occuring on the map at the same time (scouting, macro, battles, drop defense), that immediately puts it at an advantage, and many pro level strategies rely partially on pressuring the opponent until they make a mistake.
The high level strategy stuff exists on top of all this, and that's what makes SC2 so difficult. AlphaStar is really impressive, but once it understands the game at some level, I think it's inevitably going to be better than human players. Human players are going to get wins off AlphaStar by forcing tactical errors in weird scenarios, but it's not going to look like typical human games.
It’s worth mentioning the huge advantage that impossible dropship or blink micro can give. It isn’t raw apm, but perfect timing sometimes that is an issue. The management of attention is truly important
Sounds like the AlphaStar team should add some randomized jitter lag between when Alphastar issues a command and it’s executed. Would make it learn more robust strategies that were less dependent on perfect timing, and might look more like a human player.
I'm not sure how you can write all that and still say APM doesn't matter. It's clear that in order to pull off simultaneous events while juggling macro and micro, you need sufficient APM. Being a pro means you know how to manage APM, which is essentially a limited resource, while accomplishing what you set out to do. If APM is no longer a limited resource, that removes another constraint on how you play and what you can do in the game.
What he means is that even if the computer controlled at an amateur level of about 100APM, it might still defeat a pro that plays at 300APM, if it spends those 100APM more wisely. In theory it might be easier for a computer to manage multiple fronts than it is for a human. It doesn't need more APM for that.
More APM does not mean you have more attention to spend, it means you have better micro at the position you are currently spending it.
I'm pretty sure that if an AI would perfectly spread 100APM over three fronts, it would already play at pro or close to pro level.
If it plays at 3000APM, which it could do, it would beat pro's just because of micro which is less interesting.
You're going into a lot of mental gymnastics over this, which just shows how crucial APM is. There's quite simply a minimum APM required to micro, macro and handle multiple fronts at a sufficiently good level. And it's not 100 APM. And the more APM you have to spare, the more your possible decision space grows on tactical and strategic levels. Hell, the previous AlphaStar matches show what kind of ridiculous things can be done with 1000 APM. It's not just some magical number. Your armies become more effective the more APM you have.
I think the point of disagreement between you and the other commenters is a subtle equivocation over the meaning of APM, something I've found to occur not infrequently in discussions around a metric.
You seem to be talking about an underlying, unknown parameter for which APM provides a crude estimator. When you use the term APM as a proxy for that hidden parameter, people will often mistake your position for claims about the raw number they see on screen, which does not perfectly correlate with the desired attribute.
In the case of APM, there are two major sources of error that make it a less-than-ideal estimator:
1) Not all actions are counted by APM. Moving the camera, moving your eyes to look at the mini map/minerals/supply counts/unit health, moving the mouse without clicking anything, pressing hot keys for targetable actions without confirming a target; these are all things which are not reflected in APM.
2) Many redundant actions are recorded as APM. Ordering a unit to move to one location and then clicking again in the exact same spot is a big one. Same goes for repeatedly selecting and deselecting groups of units without issuing any commands.
Given the above sources of error, it is not unreasonable to believe that a player with 40 APM can defeat a player with 400, it's just less likely.
The problem with extending this discussion to an AI is that the software does not suffer from these limitations. We can measure (and in fact control) the true number of actions the system uses to play the game. Unless programmed incorrectly, an AI should not waste any actions at all, so its measured APM won't be any higher than necessary. This makes it difficult to compare to a human player's APM.
> 1) Not all actions are counted by APM. Moving the camera, moving your eyes to look at the mini map/minerals/supply counts/unit health, moving the mouse without clicking anything, pressing hot keys for targetable actions without confirming a target; these are all things which are not reflected in APM.
Not all actions need to be.
> 2) Many redundant actions are recorded as APM. Ordering a unit to move to one location and then clicking again in the exact same spot is a big one. Same goes for repeatedly selecting and deselecting groups of units without issuing any commands.
The assumption here is that there is an end-all be-all metric for APM. It isn't. Just look at the discussions around EPM vs APM to get a taste. Anybody who thinks about APM understands that it isn't a 100% accurate measurement, and it just doesn't add anything to the conversation when everybody realizes that APM isn't 100% accurate.
Further, it's no accident that AlphaStar's APM has been limited, stemming from the kinds of advantages it provides in a game like SC2 where APM is a limited resource for every player such that it lays constraints on and is related to or is constrained by all kinds of things from attention to cognitive load to strategy to micro to macro. APM is itself a metric that's incredibly important to the game, not just a proxy. The fact that it's related to so many other parts of the game is just an indicator of how important it is.
Also, it's a big jump to say that all of AlphaStar's actions are effective and not wasted. At most you can say that there's no spam, because there's no need to. Pros will spam clicks just to keep their hands warm and moving, something which an AI doesn't need to do.
I'm saying DeepMind could be better at lower APM than a human could be. Which is cool because APM we see as a physical limitation, topping out at around 350 average iirc. If you go through a players 350 clicks, how many are good tactical decisions, how many are mediocre or bad ones and how many are just the player keeping tempo?
We don't know how our attention span is limited and how that relates to tactical planning.
I don't think APM is misleading. Your basic assumption is that computers are better at StarCraft than humans. The whole point of these experiments is to prove that. Or to see whether we are advanced enough to build such a computer.
The whole point of these APM shenanigans is to make sure that the competition is about intelligence, not interface. Even without any fancy technology at all, the computer starts out with a massive interface advantage. An API is just better that a screen and a mouse. It would be really cool if we could hook up a human brain to SC2, to give the human the same interface, but we can't. So we go the other way and limit the AI's Interface.
The final step in that would be to give the AI the same interface as the human, i.e. point a camera at the screen and connect a robotic arm to the mouse. But that just creates computer vision and robotics challenges, which are probably seen as a distraction at the moment.
But such a robotic arm would definitely have a limited APM, so the current approximations don't seem to bad to me.
I think it would be very interesting to keep the full map view, but then brutally limit the APM, within like a 1-2s window.
Limiting map vision seems a kind of arbitrary limitation to me, an artifact of the particular minimap size Blizzard chose that you can't change; you should be able to change its resolution, size and placement as you like, and then face the computer on roughly equal terms (without its ability to do ridiculous micro, i.e. try to produce pure strategy games).
You also need to be somewhat even during the whole game, though, corrected for race. If you play 10apm vs somebody at 200apm, they will be able to a move at the engagements even if your burst to 400apm because the difference in army size will be that large thanks to the differences in macro.
>> If it can instantly understand and juggle different scenarios occuring on the map at the same time (scouting, macro, battles, drop defense), that immediately puts it at an advantage, and many pro level strategies rely partially on pressuring the opponent until they make a mistake.
Then you could restrict the AI to a certain amount of memory that is available to it. You could also play with latencies/exactness of the memories that is available to it.
> If I have 20% more APM than you do I beat you even if I totally pick the wrong units and build order?
Thing is, you are probably going to pick units that require more APM. Some units like the blink stalker really benefit from micro-management if you can do them quick enough. If you have enough APMs to juggle 5 or 6 immortals out of 2 warp prisms, you have a huge advantage even in front of an opponent with a superior strategy.
And in the previous rounds, AlphaStar did have heavy APM limitations but they were criticized a lot because they allowed for very high APMs bursts. Being able to perfectly micro-manage engagements of 20+ blink stalker is going to do the equivalent of doubling your unit count.
The other thing that differs from a human is that as the APM increases, the precision can suffer. Humans will misclick, will fail to select all the units they want, etc... AlphaStar will not because it essentially sees Starcraft as a turn-based game and can react perfectly to each tick.
> Could a super smart bot win against pro's even if it's limited to 20% less APM?
During the DeepMind vs Pro Players show games DeepMind was limited to about 300 apm average, but had bursts over 1500 apm (that is blinking 12.5 stalkers individually every second) which made battle very unrealistic.
An AI which doesn't misclick or toggle between buildings constantly should in the long run probably be limited to a much lower burst apm to be comparable to humans.
It makes sense to impose such heavy limitations to AlphaStar in order to have it demonstrate its mastery of the strategic aspects of starcraft.
I am looking forward to seeing it applied to a problem where we don't tie one its hand behind its back though.
There are probably real life situations (hopefully not war, although that will happen for sure) where being able to focus on 10 things at the same time and react extremely quickly without any fatigue would make an AI way more performant than humans.
Come to think of it, fatigue might manifest in thermal throttling unless we conceive its chassis better than we do our phones and laptops.
One more reason why I am saddened that SupCom is not more popular.
It would be very interesting to see AlphaSC trounce humans at this game.
The far away view that was called "cheating" in the alphaStar exib match ? In this game it is just what high level players use most of the time (the zoom level is free.. from way too close to a continental view)
Ketroc plays Master level Terran SC2. His APM is poor, he sometimes gets supply blocked, his micro isn't great. But his strategic understanding is excellent. Ketroc grasps exactly what's going on, and also how other players are paying the game, which leads to him winning at Masters level (not pro but very good) where most competitors have high APM.
Ketroc was famous in early SC2 for "that blue block is moving" a phrase from commentary of a game in which his opponent takes Ketroc's base but Ketroc's superior strategic understanding leads to an unconventional win.
https://www.youtube.com/watch?v=FbfIVf49zXA&t=530s is Day9 commentating on Ketroc doing something similar, but now aggressively, for a "FunDay Monday" (Day9 fans are tasked to play SC2 in some particular way, partly as training mostly as amusement).
>If I have 20% more APM than you do I beat you even if I totally pick the wrong units and build order
At 20%, no. Pro players beat other pro players with 20% more apm all the time, even if they pick reasonable unit comps (this is especially common when terran players do tank focus builds against zerg). At 200%, especially if it's effective and not spamming, maybe. There are videos of bots microing marines at very high apm to beat big baneling balls, which cost much more resources and are supposed to directly counter marines. Protoss players could have perfect blink micro or immortal dropping, etc.
Alphastar restricted to lower average APM than MaNa beat MaNa's mass archon-immortal with mass blink stalkers :)
He blinked stalkers at once from 3 sides of MaNa army (which was several-screens-wide). Such micro wouldn't be possible for a human player, because you would need to know to move the screen to see a particular stalkre just in time to blink it when he has almost 0 hitpoints, and immediately move the screen so you see another stalker on other part of the map to blink it, and so on.
And the bot did this, making the strategy part of the game irrelevant (blink stalkers are supposed to lose vs immortals).
Exactly how AlphaStar was restricted is controversial. Its peak/burst APM during fights seemed extremely superhuman. That the average APM was lower doesn't mean much. Insane APM will win fights even without the multiple screen thing.
APM ranges change the whole game. If you have low APM - one kind of units are superior. When APM get's higher - another unit might become superior.
That is also why amateur players copying pro-players can loose to a "weaker" unit combinations. Because when you do things at 50 actions per minute you will not get the advantages of certain unit types.
In the context of AlphaStar it was clearly visible in its first games. The AI chose ranged units and every commentator during that game was saying that AI has an inferior army. However when within the battle its APM were allowed to reach 1000 that type of unit became the best possible unit to have.
Not only unit movement (micro), but your ability to macro (manage your economy, produce new units, expand your base) will be great affected if you have low APM. In that sense, APM is very important. I'm confident I can beat any bronze league (lowest tier) player with nearly any unit composition (even when they pick a build that is a direct counter to me) simply because I will way out produce the bronze player since I'll be able to stay on top of my macro due to superior APM.
Have you seen the MaNa-AlphaStar games? The bot was restriced in APM (and had lower average APM over the whole game than the human players), but it wasn't restriced in how many times a second it can move the screen around. And it used APM limit very strategically - for most of the game it had low apm but in fights it had sometimes over 1000 APM :)
It outmicroed MaNa (one of the most micro-focused protoss players) in several games, attacking and winning from positions it shouldn't be possible to do so. In one game it countered MaNa's big immortal-archon army with pure blink stalkers. The general consensus is that immortal-archon is the ultimate counter to stalkers :) The bot was doing perfect blink stalker micro from 3 sides of MaNa army at once over an area that was like 3-screens wide.
It would be physically impossible for a human player to play like this, because moving the screen to blink each stalker as it gets near 0 hitpoints back and forth just can't be done fast enough, with 50 stalkers fighting - but for a bot it's easy.
After these games they limited the alphastar to have limited speed of moving the camera, and that bot lost to MaNa (but it was also trained for less time so it might be not a good comparison), and MaNa strategy was a little cheesy in that game - harrasing with warp prism making the bot move his army back and forth.
Starcraft 2 is as much a strategy game as a mechanical arcade game, different mechanical restrictions change which strategies work well vs which other strategies. The balance of the game is designed with human limitations of micro in mind, so when a bot plays it has to imitate these restrictions, or the game isn't fair.
With perfect micro mass marines beat everything with ridiculous cost-effectiveness.
> MaNa strategy was a little cheesy in that game - harrasing with warp prism making the bot move his army back and forth.
Even from the games where MaNa lost, AlphaStar made incredibly poor big-picture strategic judgements, in terms of where to deploy its forces. It can't seem to smartly deal with harassment.
> Starcraft 2 is as much a strategy game as a mechanical arcade game, different mechanical restrictions change which strategies work well vs which other strategies.
This.
Show me an alpha star with robotic arms and camera eyes using a mouse and keyboard. Then we’ll be playing the same game.
Or, I suppose, give humans a non-physical neural interface.
This is definitely something I want to see. I can't get that excited about a bot that clicks faster (I suspect spell units---high templar, infestors, and ghosts---are the best way to leverage that in a reliable way). What I do get excited about is: what can AlphaStar teach us to help us play better?
Another way to formulate this is: if you made AS as broken as it could possibly be by micro, what macro strategies would humans have to discover to beat it?
If this is a problem in need of solving couldn’t developers simply implement random miss-clicks or key stroke errors in a range similar to humans? Could be interesting to observe how the AI recovers.
Sounds like the chaos monkey program Netflix used to test is infrastructure by introducing random amounts of latency, switching off certain parts of the infrastructure etc
And it highlights a problem with using a game like StarCraft to measure AI - there isn't overwhelming evidence that the humans are winning by being better decision makers. If DeepMind wants a computer to win at StarCraft they can trivially make a computer win at StarCraft. The research seems to be in 'how do we handicap an AI so that it doesn't win easily by leveraging its more reliable sensors' and they are adding handicaps in a semi-random manner. That is nearly pointless; they need a different deterministic game where hidden information plays a bigger role and fast reaction times don't matter.
Ignoring APM - what if it turns out that the better of two players isn't the one who makes better strategic decisions, but the one who can issue more effective tactical commands in a short burst? We can't measure that as it isn't APM but it is likely to be a huge factor. StarCraft AI have historically been handicapped by being unable to model tight tactical situations - if machine learning can come up with any model at all that works (and we now know it can) computers are simply so much better at making quick decisions that there is little left to see that is interesting. There isn't much left to prove.
GTA5 could be a more interesting benchmark. Leave a few whole missions out for the test set, and then see if the bot can beat those missions on the first go.
> Like what is the relative importance of APM to strategy?
I wouldn't so much talk about APM, as about time-costs of information.
In real-time games like Starcraft, you have to spend time to look around the map and figure out what's changed in each area since you last saw it. The more of this "espionage" you do—and the longer it takes you to absorb what you see—the less time you can dedicate to actually doing things, making APM matter more and more.
This—presumably—won't be a problem for AlphaStar, though, because with a limited APM, it will be able to process the information it receives far faster than it will be allowed to emit actions. Think of it like a robot that could read (and absorb!) a book by just flipping through the pages at speed. If AlphaCraft only needs to just slide their viewport across the whole map as fast as the client allows scrolling to happen, in order to learn everything about what's going on, then the time cost of their information is very low, so their APM is basically irrelevant. They have all the time in the world to do whatever they like.
I'm looking forward to the presumptive second stage (AlphaStarZero? ZeroStar?) where the AI will train by playing itself. Because both sides would have equal APM potential, there's no reason to limit APM in that case. And, therefore, the game becomes a much more real equivalent to a normal StarCraft match, because once again the AI is put in a situation where the other player is reacting quickly-enough to their actions than they have to weigh "time spent learning what happened" against "time spent performing actions of their own based on incomplete information."
In the demo matches in January, AlphaStar did deploy "superhuman" techniques (that even the top players can't deploy consistently or at all). IIRC the issues were: while AlphaStar had a per minute APM budget, it could distribute it strategically. The point was made that the peak wasn't much higher than pro players, but that fails to take into account that A* has much better timing and accuracy during these peaks.
People in the comments made points about "perfect timing" and "management of focus" and I concur wholeheartedly.
> APM is an interesting question I think. Like what is the relative importance of APM to strategy?
Depends on the strategy, and the skill level you play as, and what that APM is being used for.
Among the lower 90% of play, a player with a very good understanding of strategy, and good prioritization of their actions (Spend money, don't get supply blocked, don't lose your army to a dumb blunder) will win ~100% of their matches with ~40 apm, assuming they plan to use a low-apm strategy (no fancy drop play, preference to units that don't require a lot of micromanagement.)
On the masters/pro scene, though, you need a lot of APM just to stay alive. When there's three medivacs cycling drop harass in your main and natural, while helions keep running into the worker line of your fourth base, while your opponent's army is jostling for position with yours, trying to get a good angle to land EMPs, or to siege up in a good position, you're not going to be able to physically stay alive, unless you are devoting over a hundred APM to dealing with these threats.
> If I have 20% more APM than you do I beat you even if I totally pick the wrong units and build order?
That depends on what 'the wrong units and build order' means.
Some build orders will straight up die to other build orders, assuming perfect play on both sides. Assuming non-perfect play, it depends on which mistakes are made by each side.
Some build orders are soft-countered by other build orders. They can, theoretically, hold against a 'countering' build order - but will die, if the person using them makes some mistakes, and their opponent makes none.
Likewise, 'the wrong units' is huge a bag of worms. Are zealots better units, cost-per-cost than marines? Yes, unless there's a critical mass of marines, at which point zealots become the 'wrong units' for that engagement. Throughout a game, there are ebbs and flows of strengths and weaknesses, between two players, as they hit their various tech timings/unit counts/positional advantages, and try to transform those short-term advantages into economic damage, or efficient army trades.
A big part of pro play is knowing when you have an advantage you can exploit, how much you can exploit it, how to exploit it efficiently. You need game sense for the first, good strategic thinking for the second, and APM for the third.
[1] I'm a pretty casual player, but I've had no problems hitting Master (5%) level in every season I've cared to play. I generally sit at ~90 APM, when I don't spam - and I presume that if I were much smarter, better with my timings, and more knowledgeable about the game, I could be in the 10th percentile with half the APM. The amount of extra game knowledge that pro players have, compared to me, is staggering.
When trying to solve problems, if you add constraints in certain areas, you can sometimes come up with really creative solutions you wouldn't have thought of before.
> "Restricting the bot to one screen of information at a time is great"
I'm actually curious on the details of this, because it makes for a huge difference. The pro players spend most of their time looking at the mini-map, which shows you what's happening all over the map, in areas where your units/buildings have field-of-vision. You can't see exact units/buildings on the mini-map, but you can make out unit movements as well as expansions. Not giving AlphaStar access to the mini-map would be a severe handicap.
I'm also curious if they capped "screens per second". Humans cannot absorb all there is to see on the screen in .2 seconds and then switch to the next screen. Without such a cap, limiting the AI to "just see one screen" doesn't really do much.
It would be interesting to see how the AI distributes it's views if it were capped in some significant way.
Also, can the bot only issue commands on one screen at a time?
The earlier version of AlphaStar viewed the map as one giant screen. That gave it an advantage with flanking blink micro that human players couldn't reasonably do. This was with an average pro APM restriction.
If they're not limiting AlphaStar's TTFA, then it can respond instantly to problems all over the battlefield, which is superhuman in an uninteresting way.
In any real-world application (robotic manufacturing, self driving vehicles, precision guided munitions targeting...) extremely fast reaction times are an essential and expected advantage of machines. I see AI game playing as interesting mostly because it provides a way to pit software against people without building a bunch of expensive physical infrastructure. If anyone ever builds a "real" robotic army it will surely use all the APM and TTFA advantages it can possibly achieve. In that sense, allowing the same advantages in combat simulation games is closer to a proper man-machine contest than one where those advantages are taken away from the machine.
It comes down to what you are interested in. Right now DeepMind seems to be focused on beating humans on strategy and tactics, which is reasonable, since those are the contested areas and cpu winning on APM and TTFA has been solved for long ago. To not confuse one for the other, putting limits on the later (at some point maybe even stifling itself beyond what a pro player could reasonable to) seems like an excellent way to go about it.
It's disputable whether any such limitations currently applied to AlphaStar are really working as intended. There has been some controversy over it. I saw the matches and it did seem that some of the actions taken by AlphaStar would not be possible to do by a human. Human APM is limited not just by the mechanical speed of one's hands, but also by the mental models that exist in one's brain. Muscle memory can't do everything for you. But in AlphaStar's case, it can perform every type of action as if it were done with the speed of muscle memory.
The original commenter's remarks about TTFA seems to be equivalent to what I'm referring to.
They can train a discriminator network to classify actions as being generated by a human or bot. Then aim to generate actions that can't be distinguished.
> cpu winning on APM and TTFA has been solved for long ago
Has it? I'm not a RTS player but I thought that simple bots couldn't compete with top StarCraft players just with speed. It required strategy plus speed to first become competitve (though less strategy than required with slow APM and TTFA).
Apologies, I should have been more precise: A cpu will win when you start isolating and creating a scenario to test for APM and TTFA (obviously), so if you are interested in advancing strategic and tactical AI capabilities – basically out-thinking the human – it makes sense to cap those solved parts to make sure you are winning because of progress you make on your objective.
Brood War is much more dependent on micro than Starcraft 2. Sean "Day[9]" Plott has mentioned this a lot in his videos. You can only select 12 units at a time and cannot select more than one building in Brood War. And there is a bit of latency in Brood War that doesn't exist in SC2.
The idea is to see if the bot is playing as smart as a human. To measure this, you must factor out any advantages not available to humans. This is valuable feedback when developing an intelligent agent, but that doesn't mean you can't remove those artificial handicaps once going to production with something like self-driving cars.
Edit:
Alphastar does already factor out reaction time:
“We measured how quickly it reacts to things,” Silver said. “If you measure the time between when AlphaStar perceives the game. From when it observes what’s going on, then has to process it, and then communicate what it chooses back to the game. That time is actually closer to 350ms. That’s on the slow side of human players.” https://venturebeat.com/2019/01/24/alphastar-deepmind-beats-...
That may be true in some narrow technical sense, but it was heavily disputed by the StarCraft community. If you watch the exposition games, AlphaStar has superhuman ability to micro-control its units one-by-one during critical battles, which allowed it to beat its human opponents even when its army was far inferior on paper.
Not sure why you are getting downvoted. Upon closer inspection it was indeed perceived as contrary to what was expected from the StarCraft community. (And rightfully so)
I guess the thing people are interested in is a program that can autonomously play the intelligent parts of the game - the strategy and all that. I assume Starcraft is a poor choice for that, though, because of the massive importance of micro.
For instance, imagine you’re playing a chess game with a 100 ms timer and first to exhaust their timer loses. No human will win and I could create a program that could best Kasparov trivially by advancing each pawn. There’s the game and then the input layer problem.
Precisely. Focusing on APM is kind of a red herring anyways because during previous matches there were only a few milliseconds that it spiked above human levels, and it was usually when moving workers: https://deepmind.com/blog/alphastar-mastering-real-time-stra...
Even so, this latest version has max APM limits instated to appease pro-players. Since Alphastar is forced to perceive the state of the game through machine vision of the screen, it's reaction time is already on par with humans anyways (~350 ms for Alphastar vs. ~250 ms for humans).
Alphastar did some inhuman stalker micro that the APM debate seeks to cover, but doesn't.
Stalkers have a player operated ability to instantly move a short distance 'blink' once every 5 seconds. When in a fight, you optimally let the stalker(s) taking damage soak up as much damage as possible and then blink them backwards so they can recharge shields and continue firing from behind other stalkers. They don't stop firing, so all the work the opposing force did trying to kill a shooter, resulted in no outcome at all.
That functionality is balanced by the fact it is hard for a human to time activating the abilities of many stalkers at once in time with the damage they are taking and perform the many other actions the game requires at the same time.
Alphastar can perfectly blink back stalkers with limited apm because timing things is obviously not a problem for it, making stalkers way more value-for-money than they should be and can hold off high investment attacks more cost effectively. Ultimately sc2 is a game of economy and timing so this small change gives a massive advantage.
If I'm not mistaken, the blink micro was unrealistic because that version of Alphastar played a modified version of the game. It's "screen" covered the whole map and it had no need for a minimap. Human players can do some of the blink micro that Alphastar did, if restricted to just one screen of space.
What made its micro different was that it did it consistently from flanks on several sides of an army exceeding the boundaries of a human screen. It was also notable that it did that while macroing at home, but some macro actions during intense micro is done among better pros.
The blink micro advantage should be far reduced if Alphastar is playing the same StarCraft II installation as humans now.
> Focusing on APM is kind of a red herring anyways because during previous matches there were only a few milliseconds that it spiked above human levels, and it was usually when moving workers
If you believe Aleksi Pietikäinen -- and pretty much every one of the professional players who played against it -- the claim you're repeating here is so misleading as to be fairly considered an intentional lie on the part of the deepmind team.
For example, TLO's inflated APM are presented in that chart without comment. Specifically, without the comment that his high APM counts come from a particular game context in which holding the mouse button down (i.e. a single click with a duration) is counted by the game as thousands of APM.
AlphaStar APM spiked to ridiculous inhuman levels during stalkers micro. On top of that, it controlled units that were screens apart at the same time, which is not supposed to happen.
DeepMind's refusal to aknowledge it, on top of the sketchy
and misleading TLO chart didn't do them any favor.
> A. Like human players, AlphaStar perceives the game using a camera-like view. This means that AlphaStar doesn’t receive information about its opponent unless it is within the camera’s field of view, and it can only move units to locations within its view. All limits on AlphaStar’s performance were designed in consultation with pro players.
Your assumption us correct about the show matches against MaNa and TLO that many people are talking about in here. That was not the long term goal of the Alphastar team to keep it on that heavily modified version of the game. For one thing, it meant that Alphastar needed a customized version of the game that it couldn't play on the ladder. As far as an AI challenge goes, it's also really weak if the AI gets more direct access to game data than its human opponents.
I agree that trying to play video games really well is mainly about getting "cheap" experience for real-world applications, but that is exactly the reason for why they should limit APM and reaction times.
Because in the real world, you have to get a bunch of sensor data, potentially run it through it's own neural net to recognize objects, and then feed it into the decision making system. All that takes time - most likely much more time than the 1 frame it takes for the AI to make an API call.
If the AI actually played through the same interface as humans, i.e. it simply gets the rendered image as an input, and produces mouse/keyboard inputs as an output, then maybe we should disable artificial APM/reaction time limits. But as it stands now, the AI has an absolutely massive advantage simply due to using a much better interface, which it won't have in the real world.
Knowing speed runners and the like, I can imagine they will quickly find a way to determine if the player is in fact DeepMind via some sub milisecond method.
Depends what you mean by "instantly...all over the battlefield." It essentially uses the monitor/keyboard/mouse for its input and output, rather than some nice API of the game state. Thus, like a human player, it can see things on the minimap roughly instantly then use the keyboard or mouse to move the camera and start responding.
If you watch the first-person view of top human professionals it already looks pretty instant or "mechanical," and is sometimes hard to follow for me even as a semi-competent player and avid spectator. What's the TTFA for a pro when an enemy drop appears on the minimap? I would guess somewhere around 200ms at the quickest? That would be similar to the latency of the DeepMind neural network (supposedly 350ms). And, of course, Starcraft 2 already has an approximate input latency of 200ms (so that all players can receive all other players' inputs and run them against their game state).
At the end of the day I don't see pure reaction speed as being a huge issue in Starcraft 2. Perhaps it would give a computer an "unfair" advantage in some rare cases like two cloaked ghosts running into each other and trying to snipe each other.
That's how I feel about DeepMind for sc2, it doesn't sounds like a smart AI but just an AI insanely fast ( can micro everything instantly ) which is not realistic. Also APM is a usless metric especially for a computer since a computer only do useful actions, so you have a pro player doing 400 apm and DeepMind just 120 but 100% useful.
I don't understand your criticism. DeepMind's APM is limited, and you're saying there's still a problem because DeepMind will use its APM more effectively? Well...yeah, that's a big part of what it means to play Starcraft 2 better than your opponent, whether it's a human or a computer.
This is how APM is treated in the professional scene as well. Everyone knows that bursts of 500 APM when you're spamming at the beginning of the game aren't some incredible display of skill. But sustaining a good number of useful actions per minute is incredibly important, and a huge part of the mental process in Starcraft 2 is constantly deciding where and how to invest your actions.
It's really uninteresting that the machine wins simply because it doesn't have to deal with the inaccuracy and delays of mouse clicks and selection boxes. You can say that it's a limitation of humans, but aren't we searching for evidence of high-level strategic thinking in the machine, instead of purely mechanical advantages of the machine executing copy-pasted human strategies with inhuman accuracy?
This video seems to imply that latency of the neural networks is around 350ms, which is fast but within human ranges: https://youtu.be/cUTMhmVh1qs?t=1488
In terms of APM, yes, which is what this FAQ addressed:
> AlphaStar has built-in restrictions, which cap its effective actions per minute and per second. These caps, including the agents’ peak APM, are more restrictive than DeepMind’s demonstration matches back in January, and have been applied in consultation with pro players.
So yeah, they've tweaked this specifically, although not much details as to how.
The AI they so proudly showcased last year learned to beat the limitations put in place. It was really funny.
The DeepMind team put a cap on the average APM it could perform over some period of time. Being a micro intensive game, having a perfect micro even for a short time is a game changer.
The AI just learned that by dropping it's APM low it would be allowed to have insane and inhuman bursts during critical fights and still be within the "rules" made up by the DeepMind team. It turned out it was one of the most successful way to win games.
I feel like this really undersells what they showed though. Sure, APM was an issue, but all the different and varied strategies in managed to come up with was still really impressive.
It seems like they have since focused more on the "single screen" model, with more restrictive APM limits (hopefully also limiting short peaks). I'm really curious to see how far they've come. I'm assuming the big showcase will be during Blizzcon.
It was also really unimpressive in my opinion, instead of being smarter and making broad decisions about tactics that are harnessed by some perfect unit composition knowledge, the AI won on a purely micro scale.
There are videos from several microbots such as Automaton 2000: https://www.youtube.com/watch?v=IKVFZ28ybQs that showcase what super APM can do. The DeepMind AI that beat some pro players did it by having better micro while generally having worse strategies. The DeepMind matches showed how better micro can turn a generally weaker unit composition into the winning unit composition just as that Automaton 2000 video did.
To me it seemed that the DeepMind team figured out that the only way to beat competent players was to do what all those microbots do and pump APM into godly levels. They decided to limit APM, however like you said, just average APM and have the bots explode APM when needed. The real funny match was when MaNa beat Deepmind by completely breaking the AI with generally simple drop strategies and made Deepmind look very much inept.
> It turned out it was one of the most successful way to win games.
Meh. That statement is euqivalent to "Turns out being able to calculate 400.000 chess positions per second it one of the most successful ways to win games". Given perfect Micro, SC2 is a totally different game that what we are playing.
If AlphaStar is playing at the top of the ladder it probably won't be too hard to figure out that it's AlphaStar. There's only around 15 or so players with MMR above 6000, and they tend to know who they've been matched up against, even if the opponent hides their name with a barcode (they name themselves "IIIIIIIIIIII" so you can't recognize who it is. And AlphaStar has a pretty distinct play style too.
Unless it's playing more towards the middle of the ladder where there's lots of players I don't think the test would be very anonymous. Regardless, I'm pretty excited to see what happens. Hoping to luck into a game.
Should be easy to spot based on the demo. Look for the one with way more gatherers than anyone uses, a bunch of disruptors getting amazing micro shots, and groups attacking up ramps in ways human players usually don't.
I am sure they will find a way to dampen its strength while improving it at the same time. Some ideas: reducing number of neurons, reducing neural network running frequency.
If they are using monte carlo tree search or some other minimax algorithm the standard way to do this is to limit the depth of the search tree. But not sure how they would use MCTS for starcraft. Could be they are using unsupervised learning to figure out how to represent the search nodes
The other quieter announcement is that AlphaStar can play as multiple races. The first public version just showed playing as Protoss.
There's some controversy about the AI's field of vision. A person is restricted to using the small mini map for the "whole view" of the playing area. Whereas AlphaStar can view events without a visual restriction.
But, if you're into following competitive games the broadcasted match of AlphaStar vs Lambo is pretty incredible to watch. The AI used new and novel strategies that weren't considered before.
The AI showed that Stalker can be actually a superior unit to the Immortals if they can be micro'd efficiently.
> There's some controversy about the AI's field of vision. A person is restricted to using the small mini map for the "whole view" of the playing area. Whereas AlphaStar can view events without a visual restriction.
That has changed and AI's 'view' is now restricted.
From the FAQ:
> Q. How does AlphaStar perceive the game?
> A. Like human players, AlphaStar perceives the game using a camera-like view. This means that AlphaStar doesn’t receive information about its opponent unless it is within the camera’s field of view, and it can only move units to locations within its view. All limits on AlphaStar’s performance were designed in consultation with pro players.
Feels like RTS games have not fully exploited the potential of multi-monitor setup. Was the AI ahead of the game? Although eyes can only focus on so much (screens would distract, I guess). Bigger monitors also would suffer from peripheral vision effect.
Some RTS games allow for zooming out but many intentionally limit the view as a standard part of the game. Would be interesting to see how much being able to see a huge amount of the map at once would change certain games.
From what I've seen with Zero-K (a Total Annihilation descendant) on Twitch, strong players play most of the time zoomed-out to see the whole map at once; some just disable the minimap entirely.
IANAPGM, but one can observe that TA and SC are indeed different RTS games in the sense that TA has a territory control focus (so perhaps a bit more strategic), while SC relies more on the micro/macro focus (workers, some skillshot abilities - which incidentally led to the birth of the MOBA genre).
I played starcraft at a semi-pro level and can say that this: due to the speed at which things happen and through the way it is possible to use certain hotkeys (control groups, base cycling, fkeys) this wouldn't matter at all on a high level.
I know several of the later games, and even the first games (as far as I remember) allowed two fully independent screens with the UI buttons only appearing on the first one - I used the heck out of it.
I think the real challenge will be training the humans to repel the Rise Of The Machines(TM)[0] because they're not going to play fair when they come to destroy us.
The most disappointing part of AlphaStar to me was the fact that each game was played by a completely separate AI. I was sad to see that each intelligence seemed to only be able to converge on a single strategy/unit composition. It was a failure to settle on a more generalized intelligence, as well as an indication that many of its victories in reinforcement were potentially achieved as a result of having composed units as hard counters to other compositions.
Regarding your disappointment, human player goes into a game with an intended strategy, how different is this different than selecting a strategy based AI for each game?
I would like to rewatch the replays with this in mind, but the AI did appear to transition between different unit compositions and strategies at different points in the game. I wonder the degree to which this was predetermined opposed to reactive.
It was pretty pre-detemined (at least in the matches vs Mana. The best pros can react very fast to unusual strategies (cheese), or change their strategy if the situation calls for it (eg: terran all-ins).
I agree with the sentiment of disappointment of having to make a "league" with specialised AIs. It seems to be lacking a high level reasoning and executive module, and the league acts as a crutch. The league idea still super impressive and clever, but human players do not go around in gangs where only one of them takes a seat for each match.
Having watched the livestream from January[0], the AlphaStar agent was able to create its own individual agents to play against and then learn from in the same way AlphaZero learned from chess & baduk/go historic games. In AlphaStar's case this learning period was described as experiencing the equivalent of hundreds of years of aggregate gameplay.
This allows an agent to adjust to novel strategies not seen before because Starcraft II's number of possible game states exponentially increases as the game progresses; compared with chess and go that have large albeit fixed board states. There's also a competition for individuals who build their own AI engines[1] for Starcraft II much like how Stockfish and LeelaZero compete against one another in their own respective AI chess leagues.
Nope. The starcraft AI eco-system is vibrant exactly because it's hooked directly into the game plumbing. No hacky ML stuff involved.
Basically SC1 was reversed via memory location reading and was long the staple playground for AI competitions. University teams and what not.
Blizzard knew of this and silently condoned it. When SC 2 came around they helped move things over because who doesn't want that epic AI PR.
Not 100% sure how it's achieve technically on SC2 but blizzard is definitely actively helping push this technically. (within limits - they counterweight is they don't want to help cheaters hack their stuff)
>A. Like human players, AlphaStar perceives the game using a camera-like view. This means that AlphaStar doesn’t receive information about its opponent unless it is within the camera’s field of view, and it can only move units to locations within its view. All limits on AlphaStar’s performance were designed in consultation with pro players.
I don't think that answers the question. As it can just mean the AI has to move a camera-like view (sic) but still doesn't have to use computer vision to get which unit is which, health bars, and things in the like.
It is a particularly interesting question in StarCraft because of the cloaked units, which are visible only by distorting the background (and from their sound).
Most humans will be able to identify cloaked units, but only if they look carefully, and not in heavy battle. Can the AIs see and/or identify cloaked units?
It begs the question if fair gameplay is even possible unless the AI uses machine vision against the same interface. For fair human-vs-AI games, they shouldn't allow cloaked units!
I am not sure this question is really interesting because the aim of solving Starcraft is to improve planning and strategies, not computer vision. I suppose there are much more interesting tasks train on in order to solve computer vision.
I agree. I'm not super impressed with the ability of human Starcraft players to detect and parse the meaning of health bars. That's not really done by the same part of the human brain that I would consider to be "good at playing Starcraft." I'd be fine if the AI had a clean API to tell it the units that are currently on screen, perhaps with some limit on the speed at which the AI can iterate through everything (e.g. it should definitely require a "click" to check an enemy unit's upgrades).
I do agree that detecting cloaked (or burrowed) units is a bit of an interesting case, since a simple API like I mentioned would make it trivial for the AI to detect these units. Perhaps you could do some sort of probabilistic system, or some sort of "attention" system where the AI has to choose where on the game screen to spend its limited attention, and more attention in an area around a cloaked units would increase the probability of the AI "detecting" the cloaked unit. That probably comes close to matching how humans detect cloaked units, e.g. when you're deliberately looking for an observer around your army you have an almost 100% chance of finding it if it's there.
I'm fairly certain that AlphaStar does not work off of the individual pixels. It gets a more abstract representation of the game state. At least, that's how the original AlphaStar worked.
No, there is no raw images (unless this version changed).
The github repo shows that the AI takes an Observe action, and it essentially gets data on every unit it can see in a machine friendly format.
The previous version got info on every unit it could conceivable witness, including units that were offscreen, but technically 'visible' according to the game rules. This version seems to have changed that to only return units that are actually /on/ the screen.
I don't think that's really fair to humans then. When you play a lot of your brain is just trying to figure out what's happening on your screen: confusing stack of units, confusing units that have similar design, guessing enemy upgrades on units, guessing which enemy unit has the most damage to target it first, retreating your own damaged units, noticing the blur of cloaked units, and a lot of things in the like. It seems that the AI has perfect information. Limits on FoV seems artificial as it won't impact any of that.
I really don't think that's true for high-level professional players. Sometimes, yes, it's difficult to immediately count exact units compositions. I don't think professional players are getting confused by unit designs or finding it difficult to identify low-health units. (Don't pros play with all health bars on?)
It might be "unfair" for some almost purely number-based confrontation like a phoenix-versus-phoenix battle. But are we really that impressed by testing which human happened to be better at estimating the number of stacked phoenixes? To me, that's not what's interesting about competitive Starcraft 2. I'm more interested in micro, like pulling back injured units, or choices like "it makes sense for me to force us to trade out our phoenixes right now even if I lose the battle."
IMO image recognition in computer game would not be hard to implement and it would be very pricise. So it wouldn't change anything in the end. And image recognition for computer game probably is not very useful. Also your brain is limited by neuron number, your energy source and AFAIK AI does not use those limits either. It might be interesting in a sport sense to compare human and computer which allowed to use only 10 watts of power, for example. But in practice you don't have those limits, so it's more interesting to push the bounds.
Blizzard developed a special version of the client just for Deepmind, so no, they never had to process images.
Because of that they have a problem with invisible and burrowed units - human players can miss the shimmer/shadow because it's barely visible, but their AI sees it every time. I think they gave up on simulating this stochastic human behavior.
Still, the game is complex enough that the fact that they can play it at a professional level might as well be a miracle.
I wonder in which way an ethics board was consulted for this. Is it ethically justifiable to introduce an anonymously "cheating" AI like DeepMind in the competitive ladder? After all
> A win or a loss against AlphaStar will affect your MMR as normal.
While the scenario at hand might be relatively unproblematic, at what point does an ethics board get consulted for anonymous trials concerning AI - human interaction? Is this something on the radar of the DeepMind team?
It’s opt-in, and also it doesn’t even come close to matching the descriptions in the bill.
> for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.
I'm very interested in what sort of adversarial gameplay can be used against these sorts of AIs. I've read that the chess versions all have weak points, where if you know you are playing an AI, you can easily exploit them and win. They usually seem to deal with strategies over many moves, since the AI look ahead approach cannot go very many plies deep. I suspect that once a human can break a micro spamming AI tactic he can consistently beat an AI with a more long range strategy. And perhaps this approach is in general undefeatable by an AI, since it seems that AI play can only either be short term spam attacks or long play look up tables.
The search space for short term play is tractable, but long term play is exponential, so AI brute forcing will lose out to human principled reasoning. Thus, without human input in the form of game tables, the alphazero variant AIs can only optimize for short term spamming. And the alphago/deep blue variants are always downstream from human play, since they depend on human gameplay tables for deriving long term strategy and insight.
> I've read that the chess versions all have weak points, where if you know you are playing an AI, you can easily exploit them and win.
That may have been true in the past but it hasn't been so for more than a decade. A laptop running Stockfish handily beats top human players, even if the players know a year in advance the exact hardware and software revision they will be playing against.
Kramnik, then still the World Champion, played a six-game match against the computer program Deep Fritz in Bonn, Germany from November 25 to December 5, 2006, losing 4–2 to the machine, with two losses and four draws. ... There was speculation that interest in human–computer chess competition would plummet as a result of the 2006 Kramnik–Deep Fritz match. According to McGill University computer science professor Monty Newborn, for example, "I don’t know what one could get out of it [a further match] at this point. The science is done.". The prediction appears to have come true, with no further major human–computer matches, as of 2019.
Indeed. Most “AI difficulty” settings in games involve either crippling the player or giving the computer player extra bonuses/abilities. They are “how much should the computer cheat” sliders. (I’m looking at you, Civilization) It would be great if the norm in game AI was to allow the computer player equal game resources/rules to the human player.
It's probably gotten harder and harder to dedicate dev resources to building a robust bot player for your game, given that the top tier of players are just going to play online anyway.
It's not only dedicating the time, it's also hiring/maintaining a strong game-AI team able to consistently pump out effective models at the same pace as games are released. You can't just take a regular ol game developer and expect them to make a really competitive game AI without cheating.
Even though the work is interesting I doubt you're going to be able to build a full team of reinforcement learning experts for cheap. I would guess that maintaining a 5 person team would cost about $2-3m/year.
It might be easier "tomorrow", something like universal AI, where you plug your game objects with some kind API, provide some hardware to simulate games and it learns automatically. Something like Google does now, but more universal and easier to use. Might be a good idea for startup :)
Depends on the game. Quick-paced, short-lived games like arena fps and rts games naturally develop large enough communities (or die trying) that AI-development can be brushed off in favor of letting the players deal with it.
But for games like grand strategies (eg civ), the player population doesn't typically run high enough to support only humans each match (at least partially because the games run too long), and so its more important to develop a decently sane AI.
>It's probably gotten harder and harder to dedicate dev resources to building a robust bot player for your game
I don't think so. Years ago gaming programming books were all about how many CPU cycles can you spare for AI. Literally omg the graphics are too slow we have to make the AI stupider.
These days you can throw spades of power at it (comparatively).
He was talking about justifying the business expense and time required for a dedicated team to handle AI and only AI. And then add that this generation's consoles are the lowest common denominator and their CPUs simply don't have any spare cycles for intricate AI.
For a lot of FPS and Third Person shooters the AI is normally rock dumb, which is a shame. Compare the original FEAR's AI with many of today's shooter can't see to emulate. This guy done a good run down of how it is relatively simple but the emergent behaviour makes it more challenging.
This game was made 15 years ago and every time you play the game (just like the guy says) the encounters are different. There are ways to cheese the game still but it isn't easy to do it and tbh it almost not worth doing.
I think the problem is that a lot of games have been constrained by two things consoles being quite limited compared to PCs and the industry trying to constantly sell graphics and celebrity voice acting over gameplay.
Most of the open world games except for GTA are essentially pretty stale.
I always thought this was an intentional design decision. Players react badly to being "outsmarted" by an AI, so to keep the game fun they made the AIs predictable.
No nothing about this is intentional. Designers are constrained by ability - nobody is fielding intentionally stupid AIs. Intentionally easy maybe - very different concept.
For a FPS like doom its trivial to make a "perfect" AI that never misses.
Much of the focus is on RTS though. (Real time strategy - like starcraft)
It's a little difficult to describe but practically the "quality" of a AI is very obvious to seasoned players. i.e. You feel this & there is no masking the truth easily.
e.g. AI opponent nukes my bases where maximum damage is inflicted. Meanwhile I know the AIs scouts didn't have visibility of that area. There is no way the AI could have legitimate know that's where to place the nuke for max effect.
The AI won. It didn't outsmart me. It cheated via being omniscient and having full sight of map while I'm constrained by "fog of war".
Excellent AI - killed me. Yay.
Bad game. Bad experience. Unhappy user.
Shoutout to the Queller AI mod on planetary annihilation with titans.
It would be interesting to see if it clicks on the minimap 20 times per second to keep its state refreshed in addition to issuing orders when viewing the correct screen.
They claim that adding the limit barely affected the performance of the bot. So I'm guessing you're right. Requiring it to scan through different screens is not really taking away the inherent advantage of being a computer; it just means a few extra calls.
AlphaStar is APM-limited, which means it can't just rapid-fire click every location on the map. It has to judiciously pick which places to look in the same way humans do. (At least that's how it worked for the version they showed last year.)
> It has to judiciously pick which places to look in the same way humans do
There's a lot in that statement that can mislead.
It has to "look" - that's true, AFAIK, and is super interesting. But that's not the same as a human. If AlphaStar is watching 5 points of interest on the map, there is basically no chance that it screws that up. For humans, that would involve bouncing between the minimap to commands (assuming we're not talking about hotkeyed locations), which invites lots of dexterity/accuracy issues, not to mention visual comprehension.
Do you really think AlphaStar will have issues making out the blur of a cloaked observer? (for example).
Consider some blink stalker micro - a human CAN select each unit as it's attacked and blink it to the back of the pack, keeping a steady rotation of fresh shields to tank the damage. But that is error prone, and even the best pros only do it in limited skirmishes. Not because of APM - it's not the raw clicks that are the problem - but because of accuracy and the opportunity costs of the attention. Alphastar won't have accuracy problems, and the opportunity costs of the attention are VERY different from the human costs.
I think Alphastar is a great experiment, and I am glad they are cutting off some of the brute force advantages a program has vs a human opponent, but that's not the same as saying it is doing it "the same way humans do".
I suppose if they wanted to, they could pretty easily add a click accuracy limitation as well, and force it to work around that, as with APM and map view.
My prediction is that no matter what limitations you add, some people will always reject the notion that a computer can be competitive with a human at anything that humans are perceived or expected to be uniquely good at. In chess, there were decades of resistance to the idea that computers were truly competitive the best human players in a "fair" match, even though that game has little or no physical component (I suspect this persists to this day).
For some people, either there will be endless litigation of every tiny physical difference between the computer and human player that makes it "unfair," or the premise will just be abandoned and we'll hear things like "well, yeah, computers are great at RTS games, but RTS skill isn't really a sign of true intelligence."
As the objector above, I feel the need to defend myself despite the lack of an attack :)
I think you are correct. There will always be those people.
I hope I don't fall among them - I raise the distinction about being "like" a human not because I think it's makes a good/bad qualitative difference, but because I'm hoping to avoid such comparisons. To me, far from proving it is "good" or "bad" at the game, AlphaStar is interesting for the behaviors it uncovers that are useful to humans. (example: AlphaStar overloaded workers - a strategy that had been discarded by almost all pro years before, and is now enjoying a reevaluation as a result). Paying attention to how the I/O is different matters to such elements, even if it is pointless in the comparison of "true intelligence".
FWIW, when it comes to AI I have a more Minsky-view of things (in my limited understanding) and think that we're comparing apples and oranges without any awareness that its all fruit - we only KNOW apples. I think AlphaStar already has a better understanding of RTS than I do (low bar), even if we ignored the differences. That, however, isn't terribly exciting. AlphaStar showing us new tricks we can use - THAT'S interesting. (And now I want a segmented apple, dangit)
I think philosophically the idea of humans competing with computers comes down to balancing two sometimes opposing things: A) which types of skills are interesting to test, and B) which types of skills are inherently interesting for humans to have.
For A, this comes up a lot in discussions about video game design and balance. Do we really want to be testing how good players are at detecting cloaked units or exactly counting groups of units in battles? I tend to think those aren't super interesting strategically, tactically, or mechanically.
For B, there's a reason that human competitive weightlifting or sprinting is still interesting, even though everyone knows that machines could trivially win those competitions. Of course, those aren't really tasks that are considered primarily measures of intelligence (although, see Moravec's paradox). It's damn cool to see the limits of human ability stretched.
Of course, questions about expensive gear, performance-enhancing drugs, and even prosthetics and cybernetics can already challenge our philosophy of what makes human competition "fair." We inherently want to test the inequalities of humans, both we're only interested in certain inequalities. Generally, we're interested in who can lift the most weight, not in who could take the most growth hormones without dying.
People often pick something easily categorised for these comparisons, which seem to often be tasks ideally suited for the current direction of automation. A computer will never beat a human at chess. Go. Starcraft. Super Mario. Something else with a clear and simple set of inputs and outputs, and a fairly easy "goodness" measure.
Simply thinking out loud, take a human player skilled in chess and Deepmind's chess player, and ask them both if we could teach a badger to drive a school bus, and what changes we'd need to make to the bus and the badger and the system around it; facetious as I'm being, this is the kind of situation in which I don't see AI making any qualitative inroads, which humans remain good at - massively out-of-context problems.
As an aside, on the sbject of "true intelligence" or "general intelligence"; I'm not convinced there's any such thing, and if there is, I'm even less convinced that humans have it.
consciousness (and the higher-level awareness that feeds general intelligence) costs calories. Thus, we are evolved to MINIMIZE THE NEED. We like to think of ourselves as self-aware, and we can be...but most of the time we're in a lower state. When this near-lizard-brain state runs into something it doesn't have a preprogrammed response to, consciousness is engaged. We figure out how to respond to such situations...and we shut down again.
Once we learn to break this cycle, a great many wonders and horrors will be unleashed as we wrestle with what to do with a larger quantity of time being AWARE.
I myself am terrified that I don't get bored the way I used to as a kid. I assume this is because my lower-awareness self has plenty of pre-programmed tasks to manage and my higher awareness just doesn't get activated nearly as much.
I might be remembering the wrong bot, but iirc it was limited only in average APM, and targetting pro-level average APM.
So two immediate and obvious flaws were:
1. It could still burst, with overwhelming APM during combat
2. Pro APM is not the same as bot APM, as humans have far more limited precision, and a lot of human APM is wasted on redundant behavior as a result (eg spam clicking to make sure the order goes through)
If its still the same, then the bot probably still has lots of excess APM to burn through without impacting micro performance.
> A. AlphaStar has built-in restrictions, which cap its effective actions per minute and per second. These caps, including the agents’ peak APM, are more restrictive than DeepMind’s demonstration matches back in January, and have been applied in consultation with pro players.
The version they showed in january had full map visual information, but they focused the camera on region with the most actions.
The new version is limited to one screen of information at a time as well as a 350ms reaction time. My question is if it can "click" through all possible screens in 1 ms, then click through each screen 350ms later with a reaction.
The other question is what constitutes meaningful APM. If only 20% of human APM meaningful do to redundant clicks and hotkey cycling, what is the appropriate AI APM.
This is an interesting question. I would expect that to some degree, yes it could cycle through locations of interest to sample multiple spots. Pros do this quite often, so not only would I consider this legal but perhaps necessary for decent skilled play.
Is it much of a limit for top professional players either? From what I can tell, they spend a huge portion of their time staring at the minimap anyway, to find the portion of the screen is the best place for them to focus their attention.
I guess with number of APM/delay restrictions this would be the same as what a pro player can achieve. And pro player definitely do this, they bounce over the map all the time to monitor what is happening.
Very curious how this is going to turn out... This sounds like they are trying to train a bot to actually not be perfect? Otherwise I think they would say straight out: "You'll only have the option if you are grand-master". Since they don't say that, it feels like they are probably training the bot to "play like it's at Gold League" etc.
If it could make the same types of mistakes and overlook things in the same way a human would, that would be very interesting for game creators. Most game creators would tell you: usually the hard thing is not making a good AI, but making an AI that is fun, but still loses. We've all seen that AI can usually just win any game, if you give it enough juice.
> This sounds like they are trying to train a bot to actually not be perfect?
(At least part of) The training has MMR as a label from unsupervised learning from human games so as part of the configuration they can set the agent to play like someone with a set MMR.
Lex Fridman's interview with Oriol Vinyalis on Lex's AI podcast covers this in more depth.
In January 2019, DeepMind introduced AlphaStar, a program playing the real-time strategy game StarCraft II. AlphaStar uses a reinforced learning to learn the basics of the Protoss race based on replays from human players, and later played against itself to enhance its skills. At the time of the presentation, AlphaStar had knowledge equivalent to 200 years of playing time; it won 10 consecutive matches against professional players, and lost just one.
Just tried the mobile twitch app to check this out. Apparently you can’t watch without logging in now. Amazon much want to know what we’re watching and saying as a requirement of service
I suspect it will play unusual opening build orders, like it did in the show matches last year. That would give it away earlier in the game than its micro. I'm hoping to see AlphaStar come up with some viable build orders that pro humans haven't thought of.
For the demo they were limiting average APM not EAPM. That's how it was able to get such superhuman battle micro: by playing slower than usual before and after so that the average was below the limit.
> These caps, including the agents’ peak APM, are more restrictive than DeepMind’s demonstration matches back in January, and have been applied in consultation with pro players.
It is opt-in, as it says right in the second paragraph and then again further down the page:
If you would like the chance to help DeepMind with its research by matching against AlphaStar, you can opt in by clicking the “opt-in” button on the in-game popup window.
Humans tend to think we’re adept at the games we create, but computers have proven time and time again that we’re just not fast enough to stay on top. Machines have defeated us in chess, Jeopardy!, and even the deviously complex board game Go. Google-owned DeepMind gets credit for that last one, and now it’s dominating another game: StarCraft II. After just 18 months, DeepMind has an AI that beats the world’s best StarCraft II players, and it’s not even close.
DeepMind called its Go-dominating AI “AlphaGo,” and the StarCraft-playing bot got a similar moniker. It’s called AlphaStar, and it has more than 200 years of practice under its belt. Back at Blizzcon in November, DeepMind said its machine learning platform had managed to beat the “Insane” difficulty in-game AI about half the time. Well, it’s gotten much better since then.
AlphaStar is a convolutional neural network. The team started with replays of pro matches, giving AlphaStar a starting point to begin playing the game. Through intensive training with competing models, DeepMind was able to teach AlphaStar how to play the game as well as the best human players. Over time, it whittled the AI down to the five best “agents,” and that’s what it deployed against some of the most skilled StarCraft II players in the world.
The matches actually took place in December, so today’s internet broadcast mostly featured replays of those matches. First, AlphaStar battled a player known as TLO, who primarily plays Zerg in StarCraft. However, he had to play Protoss as that’s the only race AlphaStar trains with right now. This competition wasn’t even close — despite TLO’s best efforts, AlphaStar beat him five games to zero. Next, a different AlphaStar agent went up against a seasoned Protoss player called MaNa. Some of these matches were closer, but AlphaStar still won five games to zero. MaNa also competed against a new AlphaStar agent live on the stream, and this time MaNa finally pulled out a win.
AlphaStar demonstrated impressive micromanagement of units throughout the matches. It was quick to move damaged units back, cycling stronger ones into the front line of battles. AlphaStar also controlled the pace of battle by bringing units forward and dropping back at just the right times to inflict damage while taking less fire itself. This isn’t just a function of brute force actions per minute (APM) — AlphaStar has substantially lower APM compared with the human players, but it’s making smarter choices.
The AI also had some interesting strategic quirks. It often rushed units up ramps, which is dangerous in StarCraft II as you can’t see what’s up there until you move in. Still, it somehow worked. AlphaStar also eschewed the tried-and-true tactic of blocking off the base ramp with a wall of buildings. That’s StarCraft 101, but the AI didn’t bother with it and still managed to defend its bases.
It wasn’t until the final live match that the human challenger spotted a flaw in one of the agents. That version of AlphaStar committed to moving almost its entire army as one with the intention of swarming MaNa’s base. However, MaNa was able to repeatedly warp in a few units at the back of AlphaStar’s base. Each time, AlphaStar would turn its army around to deal with the threat. That gave MaNa enough time to build up a more powerful force and take the fight to the AI.
At the end of the day, AlphaStar won 10 matches against pro players and lost just one. If AlphaStar learned from that last match, it might be unbeatable next time.
This is just silly. I'm allowed to go into my house whenever I want and you aren't. That isn't me being a hypocrite, people who own things just have more rights to those things.
It's just amusing a company that sued individuals/companies for selling digital currency for World of Warcraft and has banned accounts from various games for relatively simple bots like mining and fishing in WoW, is going to allow much more 'intelligent' software to play against people.
What's the point though? Most people won't have the resources to out-train Google's models. Maybe if every model received the same amount of CPU time to train...
I don't believe it is using computer vision to read the pixels and translate that into data, no. It will still get the information about what's on its screen from the API.
So presumably it will get the same info you or I could get from the minimap (presence of units in a particular area but no details as to what) + detailed information about anything in its camera-like view. This will have some effect on how the bot manages its APM budget, since it now needs to consider camera management.
But honestly, from the perspective of the Alpha research, getting information from the pixels probably isn't an interesting problem. They can pretty safely assume that they could build a Computer Vision setup that could get the same info that the API gives, given enough time + GPU horsepower.
Any possible inaccuracy/delay that might result from using Computer Vision instead of the API can be simulated. You could add a response delay (already in AlphaStar), or a random chance that the API delivers incorrect information from time to time (which I don't think they do).
It is not looking at the raw pixels. It gets fed a bunch of arrays, but is limited by what is currently on the screen and not in fog. The idea is that this would be perfectly doable with state of the art machine learning and would not measure our ability at making better AIs, merely add a failure point to the pipeline.
It still has potential of being hugely unfair when only receiving everything ordinary server broadcasts. As an example in World of Tanks server sends "a tree has fallen" to the client even if said tree is behind 50 other trees, totally in theoretical field of view! = there are cheats letting you know about unseen enemy movement. Then we get to previous blunder, is APM still averaged over last 5 seconds? FAQ carefully omits any numbers, merely promising "more restrictive than demonstration matches".
Personally I am not a fan of training our future slave masters how to efficiently kill human targets.
Honestly don't get the fascination with AI for Starcraft. Most of the skill when humans plays comes down to who has better micro and macro mechanics. It's not really a "thinking" game like chess or poker.
StarCraft or any RTS presents new challenges that were not covered by previous games. Basic things such as the massive action space meant that the previous “tabula rasa” approach was abandoned in favor of imitation learning to get the agents off the ground. Micro and macro are meaningless if you cannot put a build or semi-cohesive game plan together; a skill we take for granted among human players. That’s where I think the real interesting parts come about: crafting build orders, reactions to other builds, and new tactics that may emerge. You can tell that this is what the researchers are trying to achieve by limiting the throughput in terms of actions per minute.
Watch some of the games and it might change your opinion. The thesis that "AI in Starcraft will only win via improved mechanics" is false - the AI was making some fascinating decisions / fundamentally different meta strategies.
Or watch [this game](https://www.youtube.com/watch?v=vUfwb4nOL84) between the top StarCraft AI (outside of AlphaStar) versus Serral, one of the top humans players. Unlike AlphaStar, this AI is not APM-limited; IIRC the top bots tend to play with about 100,000 APM, compared to 400 for the top humans. Serral won easily, despite the AI's vastly better mechanical skill.
Figuring out good timings, gathering intel, harrassing, doing big drops, when to invest in attacking with a gimmicky unit, etc. are all pretty big strategic decisions. A lot of people don't notice these decisions being made when they watch pro players play since they are the composition of many actions that, yes, at the end of the day are building units and moving them around. But that's like saying chess is all just moving pieces around too
I can see how micro might not be interesting for people more into strategy games like chess but the macro is strategy. Sure Starcraft may not be best for that (I much prefer Ashes of the Singularity for example for a more strategy focused RTS).
If it were just a matter of hand speed, bots using uncapped speed (tens of thousands of actions per minute) would be beating humans years ago. But quality matters way more than quantity. It's a hard problem.
I got to masters on the 1v1 ladder... there's basically a handful of build orders and tech transitions that everyone does and most of the skill difference is mechanical execution.
I believe you, but "got to masters" doesn't mean anything. Humans suck at Starcraft. Just because machines have mostly sucked even more doesn't change that.
Real time strategy games are peculiar because they are played mostly moment to moment in a very reactive style with very little medium or long term planning (beyond the next batch of units in production). So a neural network with a tiny bit of memory should be able to play that by essentially learning counters and a few harrassment and attack tricks. Sure, there's a bit more involved, but when you look at it, any RTS strategy decomposes into a lot of short term behaviours that are sequenced based on momentary triggers.
A more interesting challenge in my mind is a game that focusses genuinely on longer term planning, where actions taken at a certain time lead to results 10 to 15 minutes later in game that is simulated in a continuous fashion (that is, definitely not turn based and with no way to discretize it perfectly into turns). The loss of a turn structure makes applying min-max-search based strategies hard to apply - each side can make any number of moves in sequence. Add non-discrete moves (e.g. turn unit with a certain turn rate for an arbitrary amount of time, then move forward for another arbitrary amount of time) and the search space becomes vast and unstructured, potentially even unbounded. Solutions to that should be very interesting.
> Real time strategy games are peculiar because they are played mostly moment to moment in a very reactive style with very little medium or long term planning (beyond the next batch of units in production).
This is simply not true at a high level of play. Timing attacks, base expansions, and tech transitions involve both medium and long-term planning.
Base expansions and tech transitions as actions are all triggered by certain momentary events. This is simply due to the fact that each of necessary actions for that have prerequisites. And a neural network will simply learn long term strategies as sequences of trigger responses without any actual long term memory involved.
How big is the decision tree for tech? About ~100 nodes total? This is quite small and handling that is not so impressive. That's roughly one move in a mid game chess position (and you'd need to think two or three moves ahead to be moderately successful at that game).
Starcraft2 definitely needs long term planning. It may not look like that to an outsider because a good caster decomposes that into actions, consequences and reactions. But to be able to play competitively you need a long term plan.
The momentary triggers you mention are the distilled knowledge of long term planning and experience. A player executing a baneling rush is a long term plan. Then that player scouting the opposing terran player building a bunker and two barracks to defend can make him to decide to double expand instead of going forward with his baneling attack. This may seem re-active but in truth those decisions depend on long term planning. Pro players can make this long term planning calculations quite fast, the decisions may even appear re-active, however they are based on the long term calculation of the most probable path to victory.
>Real time strategy games are peculiar because they are played mostly moment to moment in a very reactive style with very little medium or long term planning (beyond the next batch of units in production)
Have you ever played chess? Where RTS gameplay is exceedingly short term, these turn based games rely on long and careful explored sequences of moves for successful play. This deep planning is simply not present in RTS gameplay.
Both chess and starcraft have great depth and both require short and long term planning. The fact that you can't grasp it speaks more about your own lack of insight than the nature of the games.
I can grasp the differences just fine, thank you very much. Chess requires you to think several moves ahead. This is why it can take minutes until a player makes their next move. Try this in Starcraft! The game instead requires you to switch to a conpletely reactionary style of play where all you can do is make split second decisions based on huge tables of prepared heuristics in your head.
With all due respect, I think you're mistaken. I think you are assuming a game that doesn't allow long periods of pause must be completely reactionary, but that's not true. I've been playing and watching competitive starcraft for well over decade. In competitive starcraft you need to master both micro/reactionary strategy, as well as the ability to predict and respond to your opponent over medium and long run horizons.
I believe that you've never played chess or anything similar. If you did, I'm certain that you would not say what you say so easily. RTSes are always played using heuristic, experience based decision-making on the spot. I think you're mistaking calling back on past experience with actual strategic thinking and deep planning. You cannot play purely turn based games like chess that way at all.
I think part of what this comes down to is you seem to exclude any game with a faster-paced reactive component from qualifying as on the same level of chess.
It also has much more realistic limits on its capabilities. Restricting the bot to one screen of information at a time is great. There's not much detail on the APM restrictions, but "designed with pro players" is a good sign. Hopefully they've designed some realistic restrictions more complex than just a cap on APM.
Of course the anonymous nature of the testing is interesting as well. Big contrast to OpenAI's public play test. I guess it will prevent people from learning to exploit the bot's weaknesses, as they won't know they are playing a bot at all. I hope they eventually do a public test without the anonymity so we can see how its strategies hold up under focused attack.