I once trained a net purely to predict the location of the next move - the net was given literally no data about the win/loss outcome of the game or the score or even that Go was a 2-player game with some objective. Just: see a pattern of stones, identify a location on the board. See another pattern, identify a location.
Later when inspecting the net, I found within it a channel that computed who was ahead or behind in the game and whether by a lot or a little. Noisily and imperfectly, but definitely that's what it was doing - visualizing it in many board positions you could see it consistently always assigning positive weights on territory of one player, negative weights on the other (including correct handling of life and death for dead groups!), just prior to a layer that would sum/average these values across the board into one value before feeding into further processing later.
So without being told anything about winning/losing or the score, purely in service of learning this complex and opaque mapping from "pattern of stones" to "location on the board", the neural net on its own internally discovered and learned to compute a concept that directly corresponds to what humans would name "being ahead or behind". Presumably this is worth computing because humans on average play differently when "ahead" versus "behind" so computing it helps you make better predictions, but the net wasn't told such a concept existed - it came up with it on its own!
There is no doubt that encoded within a strong net's weights and activations, whether trained on human data or trained purely in RL, are many other strategic concepts being computed and represented in different ways, many of them close enough to human concepts to be recognizable if only we had better ways of finding and extracting them. It is just that our current tools are not up to the task (and for the same reason that our tools are not up to the task of deciphering the electrical impulses in a human brain in all but the crudest ways - the task is hard!)
It's also a bit questionable to think of deep-learning-based AI agents as evaluating things "solely numerically". This is probably, as best I can phrase it, confusing the "software" of the weights and activation patterns that encode how a neural net is responding to what it observes with the "hardware" of the fact that we just so happen to be representing these values numerically and crunching them via heavy GPU arithmetic. The "hardware" isn't really fundamental to what the "software" is doing at a conceptual level and will likely lead you to the wrong intuitions about where the strengths and weaknesses of neural nets and deep learning are.
A better way to think of deep-learning-based AI agents is that when properly trained, they tend to evaluate things in a fuzzy and *analog* (i.e. non-digital, non-numeric) way, based on variously learning complex pattern recognition, high-level holistic features, or other concepts depending on the task, at the same sort of level that human instinct and snap judgment is capable of learning. With of course many non-human-like quirks (most prominent when dealing with corrupt or out-of-distribution or certain kinds of "artificial" data), but still, this is the most useful baseline intuition to start with.
Well with a human pro, even with no shared verbal language, if the pro starts playing out something that looks like a tewari, or adjusts some stone positions to show how the same move works (or doesn't) in some alternate situations, I'd expect to sometimes be able to follow along. I don't know of any Go AI that currently does such things to explain itself. (wink)Kirby wrote:If you're in a room with a human pro go player, and you don't share a common language to communicate, is it all that different than reviewing with computer AI?
In both cases, the only explanation you can get are the sequences they recommend. No explanation can be given since there is no shared language but the moves themselves.
Still useful, in my opinion.
