Life In 19x19

Hi Calvin,

Could you elaborate a bit more about conditioning ?
( For example, the difference(s) and overlap(s) between
what you mentioned about 'knowledge' and 'conditioning'. )
Thanks.

EdLee wrote:Hi Calvin,

Could you elaborate a bit more about conditioning ?
( For example, the difference(s) and overlap(s) between
what you mentioned about 'knowledge' and 'conditioning'. )
Thanks.

I can take an example from another discipline first, which I think helps because if I start with go examples it's too easy to get into details that are go-specific.

Let's consider a novice musician learning her scales on an instrument. She can learn enough theory about how her instrument works and learn enough musical notation to be able to generate the whole circle of fifths and translate those into fingerings on her instrument. This knowledge is very powerful because in theory it can enable her to play anything.

In reality, by itself it enables her to play almost nothing, because it takes time and practice reading the music and playing the scales. This practice is example of conditioning. With proper conditioning, it becomes possible to play things without thinking; e.g., develop muscle memory.

Back to go...

In go, conditioning begins with the very first game. The player needs to start paying attention to things that have artificial meanings specific to go, such as liberties. Just being alert enough to notice when liberties are getting short be can challenging for most beginners. I think Bill Spight posted some time ago that he was thinking about writing endgame stuff for beginners, looked at some 20k+ endgames and decided that the most important skill they need is seeing damezumari. CGT isn't going to help you much if you can't spot a shortage of liberties.

Conditioning can get you pretty far if the feedback is good. AlphaGo is almost all conditioning, after all.

The challenge is that not all conditioning is positive. What is repeated can be become habit whether it is good or bad. Knowledge can help one recognize the difference, which is why telling people to play and do nothing else has problems.

Hi Calvin,

Thanks for the reply.
Would you think it's fair to say reinforcements ( good or bad ) and repetitions ( good or bad ) are a big part of conditioning ? In other words, mass practice ?

EdLee wrote:Hi Calvin,

Thanks for the reply.
Would you think it's fair to say reinforcements ( good or bad ) and repetitions ( good or bad ) are a big part of conditioning ? In other words, mass practice ?

Sure. Not all reinforcement or repetition is good, of course.

EdLee wrote:Would you think it's fair to say reinforcements ( good or bad ) and repetitions ( good or bad ) are a big part of conditioning ? In other words, mass practice ?

the word "conditioning" comes from the literature on behaviourism, as in "conditioned reflex". A reflex is an action taken without (conscious) thought, ie without reflection! - which makes it an interesting choice of word...

the most well-known example of an innate reflex is the knee jerk; but there are plenty of others - we blink reflexively when dust enters the eye; and when a lovely stranger enters the eye, other reactions happen automatically...

a conditioned reflex is one that is not innate, but learned through repetition. Dogs have an innate reflex to salivate when they smell nice food; the most famous example of training a conditioned reflex is Pavlov's training of dogs to salivate when they heard a bell ring. The training was achieved through repetition of associations of bell and food.

Alphago plays by conditioned reflexes (her learned policy net), adjudicated by her (also learned) evaluation net with the help of Monte Carlo statistics.

"Reinforcement learning" is another name for "conditioning".

Practice is repeated conditioning, but practice does not make perfect; this applies to all behaviours, but maybe easiest to understand in golf: if you don't learn to swing properly in the first place, you will condition yourself by reinforcement learning to make the same mistakes over and over again.

If your piano (Go) technique is basically flawed, all the scales (practice games) in the world won't help - they can even make it worse!

Propaganda takes advantage of subconscious conditioning, as the Catholic Church recognised.

It makes no difference whether or not propaganda is logically sound or empirically true, just say it often enough and the policy nets in the heads of the congregation will incorporate it into their Pavlovian knee-jerk mindsets.

To put it another way, we are susceptible to becoming creatures of habit... unless we stop and think. Alphago doesn't need to think, because she can simulate deep and wide enough to avoid any traps Ke Jie might try to set.

Players 25k-10k for more than a few months are 25k-10k for one simple reason: they don't read books.

Calvin Clark wrote:

EdLee wrote:Hi Calvin,

Thanks for the reply.
Would you think it's fair to say reinforcements ( good or bad ) and repetitions ( good or bad ) are a big part of conditioning ? In other words, mass practice ?

Sure. Not all reinforcement or repetition is good, of course.

I felt like I understood what Calvin meant from the beginning when he described conditioning, but looking into it further, there's some interesting research that's been done in this area. I've listed a few links below ([1]).

What sticks out to me is the contrast between what is referred to as "classical conditioning" vs. "operant conditioning". As I understand, classical conditioning is an *involuntary* response to some stimulus. An example given in one of the articles describes how kissing someone might produce involuntary reaction (e.g. accelerated heart beat, etc.). If, every time you kissed a particular person and had this involuntary reaction, you also heard a particular song in the background, then later, even if you are not kissing anybody, you might experience the said involuntary reaction simply from hearing the song (which is independent of the involuntary reaction, until you've been conditioned to realize otherwise).

In contrast, operant conditioning appears to be learned *voluntary*/*conscious* behaviors in response to some stimulus. So for example, if every time I study before a test I get an A, and every time I don't study I get a C, I will be conditioned to learn the behavior that is producing the reward that I want (i.e. studying for the test).

---

I'm trying to consider which of these types of conditioning is best suited toward learning go. Maybe they can both be useful. You can split things up into the individual components of conditioning:

stimulus - Maybe this is the win/loss result of a game. Or for some people, maybe it's killing/losing a group of stones. Some stimulus might produce a positive or negative feeling, based on the moves you make in the game.

voluntary action (operant conditioning?) - The moves that you make are obviously voluntary. Maybe the act of studying itself can be voluntary, too. In this sense, if the positive and/or negative stimulus is strong enough, perhaps you can learn to play good moves and/or study if your mind draws a connection between the given behavior and stimulus/result.

involuntary action (classic conditioning?) - Perhaps some sense of focus and/or emotional state can be achieved, which is not necessarily voluntary. Maybe an elevated pulse and/or sense of euphoria from capturing a group might result from the stimulus. Or maybe a higher level of concentration can be achieved under certain environments. For example, if you always play a game of go in a particular room with particular lighting, and can achieve some degree of focus under that environment - maybe you're trained to focus and/or concentrate in that type of a setting...

Anyway, I don't know much about this, aside from the 10 minutes I spent reading these articles. Maybe somebody with more experience in psychology could give their thoughts.

[1]
http://study.com/academy/lesson/classic ... mples.html
https://en.wikipedia.org/wiki/Operant_conditioning
https://www.learning-theories.com/opera ... inner.html

Kirby wrote:I'm trying to consider which of these types of conditioning [classical or operant] is best suited toward learning go.

if ever there were a word more abused than used, it would be the word "classical". Why, there are even folk who talk about "classical" AI

That aside, the two "types" are not really two different classes, since the mechanism of neural learning is the same - Pavlov's inoperant dogs learned to associate a bell ringing with food, and Skinner's operant rats learned to associate one lever rather than another with reward. In both cases, they were subconsciously learning associations, even if the rats were consciously pressing levers.

Skinner's work caused an enormous amount of chatter, and it's still causing it, particularly in regard to whether punishment negatively reinforces behaviour. For example, there is no statistical evidence that prison or flogging negatively reinforces criminality. Consider the case of "The Mutiny on the Bounty"...

You could test it on yourself: every time you realise you made a bad move, give yourself an electric shock or stick a needle in your eye - then, after several million trials, will your Go behaviour have improved?

Kirby wrote:The moves that you make are obviously voluntary.

are you ready for a surprise?... they aren't!! New evidence from new technology for monitoring brain activity demonstrates that even when making what we think is a voluntary movement, the brain has subconsciously decided to make the movement BEFORE we become consciously aware of it!
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3458240/

djhbrown wrote:New evidence from new technology for monitoring brain activity demonstrates that even when making what we think is a voluntary movement, the brain has subconsciously decided to make the movement BEFORE we become consciously aware of it!
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3458240/

I read that in a book by Dick Swaab. Now that claim is either false or trivial:

- surely SOME of the actions which we think to have made consciously were actually a result of subconscious decision making and rationalized with some conscious narrative after the facts
- but surely not all!

My default experiment with free will is to make a decision that depends on no previous event nor has impact on the future, such as whether I'm next going to put up 1 or 2 fingers. There's no way any system can predict what I'm next going to do. The decision is completely arbitrary. There can't be any predestination.

You could argue that, the less it matters the freeer your choice, but free will and a conscious decision making mind exist.

Knotwilg wrote:whether I'm next going to put up 1 or 2 fingers. There's no way any system can predict what I'm next going to do.

but you are so predictable, because you always put up 1 (US) or 2 (UK) fingers to my posts.

Of more relevance to the original question, is whether you can, by studying the writings of experts, learn to understand better than you could purely by introspection (or by playing with yourself); your thought experiment illustration demonstrates that your subjective impression of your personal volition is contradicted by objective scientific investigation.

Yes, it's better to think something through, and you can consciously choose to exercise your (imaginary) free will to think - but the thinking itself is subconscious - you only become aware of its results after it has happened.

Have you ever made a move in Go and been unable to explain why you did it? It happened to Haylee:
https://www.youtube.com/watch?v=yTi6R-t ... a3Hl1X_v-S

As Christoph Koch says, "I choose to not have free will"

PS re-reading the above, i see it's not a satisfactory answer, because it doesn't adequately explain either the phenomenon of consciousness or the subjective experience of free will.

Both are subjects for which science still does not have clear and simple explanations, unlike the simplicity of the theory of, say, quantum mechanics!

But in any case, as far as learning Go is concerned, they are side issues, since we can't do anything about them, whether or not we subscribe to this or that theory of them - although they do have significant implications for the purposes and methods of jurisprudence.

In my videos about mental imagery in Go, i frequently use the analogy of the mind as an iceberg, with the notion that conscious reasoning is the little bit above the surface and subconscious (conditioned) intuition the greater part below.

However, i am starting to think that that image is inaccurate, and that the relative size of the conscious mind is more like the mere tip, and that even reasoning is done subconsciously.

Be that as it may, it doesn't really help to resolve the issue of theory vs practice.... but maybe there is a simpler answer:

you need both!

Carlos Santana says he is a self-taught musician, but maybe that only makes him even more extraordinarily brilliant.

Sorry to necro this thread, but I find it odd that my use of the word "conditioning" somehow resulted in people invoking Pavlov and Skinner. Those are completely different things and not at all what I had in mind. I haven't thought about those guys in decades.

I mostly meant practice, though suppose I cannot rule out the possibility that one's go might improve by spending a few months in a Skinner box getting fed chocolate for making good moves.

"The more i practise, the luckier i get" - Gary Player.

Fred Astair, strolling along Fifth Avenue, was stopped by a tourist.

"Excuse me sir," said the tourist, "Can you tell me how to get to Broadway?"

"Pratice" replied Fred.

However, those who do not practice their mistakes do not get to reinforce them.