Applied value of research

[RobertJasiek]

Quotation reference:
viewtopic.php?p=147270&sid=3d535a100a3b73559dcad93ced89d22c#p147270

The value of research, building on top of prior work, cataloguing, pushing the overall knowledge a step further has a tremendous value not only for the theorists, but also for practical applications. Maybe not always immediately, but surely later. It is the general effect of fundamental research.

has it been actually demonstrated in the field in question?

Yes.

E.g., you do not ask any longer (as you did in the 90s, doubting it could ever be clarified) whether the rules or life and death are like chicken and egg, but now you know that the rules come first and life and death are derived from (or within) them.

what if Go is different, what if in Go formal research does not bring much fruit?

The only limiting factor to "much" is the small number of researchers. Considering how small it is, go researchers are incredibly efficient. In the 90s, almost all were wondering whether ever programs would reach 1d, now they are counting down until they will be 9p.

Dave Dyer, Thomas Wolf and I have completely listed different classes of nakade and so identified or clarified classes previously neglected easily.

You keep saying 'eventually', but it seems 'eventually' never comes...

Not if you keep talking me so much that I (or anybody else) never finish getting that definition of nakade right;)

When do you expect to see measurable practical results due to all this research? In general.

In the past. Measure, e.g., the strength increment of programs during the last 20 years.

I am only worried when you will begin to admit that success of fundamental go research is not a matter of the future, but has already occurred in the past.

The fact that I look at other similar fields. For example: chess. I know a lot of formal research has been done in chess, especially in the old Soviet Union, tremendous amounts of money have been spent, and yet it did not seem to have contributed very much.

People refusing to apply it stay weak, while programs willing to apply it have beaten even top chess players;)

it seems that the best and fastest way to learn chess is still the old-fashioned way of rote memorization, examples, tactical problem solving, and simply experience.

It is a myth you and others like to cultivate, but it is fundamentally flawed. Learning by examples alone never causes great strength improvement. It is always required to be accompanied by understanding, so that the learning by examples (even if done subconsciously) knows what to calibrate for. No non-go player without any go knowledge besides the rules becomes strong by simply watching positional patterns of examples, quite like nobody discovers he laws of photosynthesis by looking the shapes of leaves and nobody becomes rich by appreciating the art on banknotes and coins.

is this research only for the sake of research

Is there any researcher doing research for nothing but the sake of doing research? Research is always also driven by higher goals, such as finding the world formula or proving its non-existence.

I seriously hope you are not going to roll out your formal definitions when teaching

It always depends on whom I am teaching. Beginners: of course, they are not taught the (draft of a ) definition text of nakade. Researchers: of course, they are shown it.

What if the traditional method is the best and most efficient?

You use a wrong premise, see above. Never is it used without also some form of conveying understanding.

What do we gain by putting so much effort ad resources into developing alternate methods.

Uh, "we"? Hear, hear;)

Maybe if you put all this effort into studying Go the traditional way, you would have been a top pro by now?

In that case, I would still not even have started to play go.

From my experience of improving from 10k to 5d, the traditional way was about one 10000th ~ 100000th as efficient as learning from general knowledge. I know, because I had to study 10000 to 100000 example moves to find a principle, whose textual reading would have spared me the effort. This is so for many principles / methods / concepts I had to discover the hard way, because they were not available in writing before.

We cannot say: what we do is extremely valuable and/or important.

You cannot so, because you do not want to say so. I say so, because, e.g., I see the extremely valuable and important results of the past. E.g., the extraordinary strength improvement of programs.

we are faced with a lot of effort to derive at these new methods (true, not my effort, but still effort) - so in principle, some kind of cost-to-reward analysis would be interesting.

Learning by examples only: neural-net programs found out that passing is not such a good move. Learning by applying research results: programs are mid dan range now.

What I question is you repeatedly stressing how important and valuable it is what you do. I am not saying you are wrong, I am just saying this has not be demonstrated yet.

You mean my formal or other go theory research measured in playing strength improvement as a consequence of its application? If you ask in the teachers forum or outside L19, I may give you an answer.

[Phoenix]

I've got laser sights and one bit of that post caught my attention. You were talking about how it's only a matter of time (there will also be much work involved, of course) before Go programs become 9p.

Now I've noticed that even (read: especially) at the top level, pros tend to handle certain situations with 'fighting spirit'. They'll play the 'strong' move because their pride will not let them back down, and they will do everything in their power to get a good result out of it.

These moves usually can't be dismissed as overplays and they happen regardless of time limits (i.e. they are not symptomatic of short times with less chance to be 'punished').

So in this future where Go AI will rival or surpass the top human level, do you think the AI will play these kinds of moves? Or do you think they will play a very sober and clean Go and react to 'fighting spirit' with a calm that will irk and confuse its human rivals?

All other factors being equal of course.

[RobertJasiek]

Programs will show what we perceive as fighting spirit. I saw it on 13x13, where the program played an unexpected early move with a global meaning, as the program then demonstrated throughout the rest of the game. Current programs "know" tenukis and justify them.

It is of course possible that weak player programmers do the same to programs overall what they did to their openings. Instead of creative center openings, the programs are forced by databases or whatever back on the traditional path. The same kind of programmers could force programs to play unexciting Lee Changho style.

But if the programmers tolerate (and learn to understand the quality of) their programs' ideas, they will show fighting spirit.

[Bantari]

Hi Robert,

Instead of going through your post sentence-by-sentence, let me just concentrate on a few points you make which I am uncertain about. I will address some specific minor points separately, but here is the gist of the argument, neatly organized in points and numbered for you.

1. Computer Go/chess

Yes, I fully agree that formal research into these subjects added a great deal to the advance in computer chess/Go over the past few decades. Putting aside the question of it being good or not (personally, I do not see it very beneficial for chess in general, although there are good points to strong computers) - I think we can safely to agree that humans and computers 'think' in ways that are fundamentally different. What's more, their strengths and weaknesses might be fundamentally different.

What I am trying to say here is that what is good for one might not be good for the other, and vice versa. If the main argument for the practical value of the research you can give is: look how well computers play - while being much more modest about any practical benefits to human players - then maybe this is all there is to it? I mean, I don't really believe it, I think there is much more to what you guys do, but what? I just don't see it.

2. About research in chess

You are correct that chess programs benefited from the formal research into chess theory. Or at least - I think so. Not being an expert myself in this field, I cannot say for certain, but would be very surprised if you were wrong on this count. However - I reject your premise that 'humans stayed weak' because they failed to apply the same methods. Humans tried, tried very hard, and it simply did not produce good results. The traditional methods produced much better results.

As I said above - humans and computers are fundamentally different animals, a lot has been written about the differences in the way information is processed by these two entities, for example, and I am sure you are familiar with some of it. They both have different strengths, and different weaknesses. It stands to reason that methods needed by one different from methods needed by the other.

This is also why I think your argument about Go computers is not convincing. Fine, so computers got stronger. Will this help me, will this help the next wave of players? Other than giving them a computer to play against - and create a whole new category of on-line cheaters?

In direct, measurable effects - it seems to me that Hikaru No Go, for example, had a much larger impact on Go than all this years of your research. Maybe, if you really want to work towards the benefit of Go, you should start drawing cratoons?

3. The early state of things

I can agree with you that the small number of researchers and the early stage of the research might be an issue. You are making and argument that with more research (more time, more people, or both) practical benefits will start appearing. I also believe they will... but I have to admit this is only a personal belief, and I think this is all you can have as well.

But all in all, I can say this is the best argument you give, and I certainly accept that.

4. Traditional methods and understanding

You seem to imply that I think traditional method does not include understanding. I have never said that. Of course understanding is necessary, but the difference to me is how is it attained and how is it 'stored'. Here is how I see it, and I might be totally wrong here, its just the picture I have.
- Traditional approach: understanding attained by implicit methods (examples, experience, etc.) and stored (and applied) mostly subconsciously.
- Forma research: understanding attained by formalized definitions and theorems and stored mostly in books and whitepapers... the practical application is uncertain, this is what we are discussing.

The question I am rising is - which is the more efficient way for a human player/sportsman to increase their playing level. I believe it is the traditional method, you believe its the formal research (once it takes off the ground with more work being done, some time in the future.) But I certainly do not know for sure, and it might be that you are right and I am wrong. All I am saying is that I don't see how you can be any more certain.

Consider some other endeavors:
- Golf: Understanding the formal physics behind a swing vs. hours of practice.
- Driving: Conscious understanding of involved engineering vs. fast reflexes and experience.
- Beer drinking: Knowledge of chemistry necessary to create beer vs. night at a pub with a bunch of good friends.

[Bantari]

Is there any researcher doing research for nothing but the sake of doing research? Research is always also driven by higher goals, such as finding the world formula or proving its non-existence.

From my lifelong experience with researchers, both personally and professionally, I can attest that their goals are usually much less noble, let alone honest. Research is done to produce papers and publications, which in turn are desirable for all kinds of mundane reasons. Getting a (larger) grant, securing an invitation for a conference, keeping your office or your job, one-upping the guy in the office next door, stroking one's ego, and so on... What's more, a lot of the research produced these days is either flawed (often intentionally) or even bogus (sometimes with the author's knowledge.)

Of course, there are exceptions, but not as many as people might think.

PS>
The above is my personal experience, so if anybody disagrees - I am fine with that. No offense.

[Bantari]

In that case, I would still not even have started to play go.

This may be so, and I certainly cannot judge that. But, does it tell us something about *you* or about the method? In other words: is it *you* who is different and needs a different approach, or is it really the fault of the method?

Conversely - all these people who got much much stronger than you, sometimes in a much shorter amount of time - are they 'special'? Or are you 'special'? Or are both groups composed of outliers and the jury is still out for general public?

Personally, I never knew any pro well enough to judge if there was any 'specialness' about him/her.

For a lot of interaction with you over the years, I can say that the way your brain seems to work is different than most regular people I know - this is why you so often get into arguments with pretty much everybody else, about any number of things people understand by you don't - remember the debate about 'sportsmanship' for example? I am not trying to dig up old bodies here, but I think it is a great example: something that was natural and inherently understood by almost everybody else was very hard to understand and process by you - you felt the need to either formalize it within the rules or dismiss it altogether. There are more examples like that.

Anyways - this all leads me to believe that you *are* different that most people, for better or worse (there is certainly no judgement on my part attached to it, so please - do not feel slighted.) Which, in turn, makes the arguments that you benefited from something much less convincing in general sense.

The fact remains - millions of people learned Go using traditional methods, and a large chunk of them got to be very strong, stronger than you and me combined. So that fact that for you personally it would have been impossible (or at least - much harder) to tread this path does not have to mean that the path is inferior.

[RobertJasiek]

what is good for one might not be good for the other, and vice versa.

There are things that are good for one but not the other. There are other things that are good for both. And yet others that are bad for both.

If the main argument for the practical value of the research you can give is: look how well computers play - while being much more modest about any practical benefits to human players

It is not meant to be the main argument, but just an important example, because every program has profited a lot. In contrast to human players, for whom we cannot say that everybody would have increased his playing strength by a dozen ranks as a consequence of formal research. Human players show a greater resistance (or call it lack of ability) WRT to applying new insight. (This is explained easily: computer go research profits from brute force abilities of computers, while human beings are very bad at brute force in comparison.)

Examples of practical benefits of formal research for humans I have stated several times in the past, see there.

I don't really believe it,

Even worse, you also forget examples stated earlier.

I think there is much more to what you guys do, but what? I just don't see it.

We, when acting as formal researchers?

Read and understand our papers etc. Apply it your games. Of course there is more: the optimists among us go for the grand plan of solving go completely. I encourage you to live for another 400 years, just so that you can see what you do not believe yet:)

so computers got stronger. Will this help me,

Only if you use their output of teaching, analysis, databases etc.

it seems to me that Hikaru No Go, for example, had a much larger impact on Go than all this years of your research.

Comics have brought new players to go, but research has contributed to make them stronger.

- Traditional approach: understanding attained by implicit methods (examples, experience, etc.)

Not only implicit methods! Show me the traditionalist who has never heard the explicit advice to defend a weak group; stronger than 10k such a person does not exist.

[RobertJasiek]

all these people who got much much stronger

In case you have not noticed yet: this is an inhomogenous group having traditionalists versus reasoning freaks.

For a lot of interaction with you over the years, I can say that the way your brain seems to work is different than most regular people I know

You know the wrong people:)

The fact remains - millions of people learned Go using traditional methods

Uh, we had that discussion in 1997. Since then, things have changed to a shrinking fraction of traditional and a growing fraction of reason-orientated methods. I am not surprised.

[Bantari]

Uh, we had that discussion in 1997. Since then, things have changed to a shrinking fraction of traditional and a growing fraction of reason-orientated methods. I am not surprised.

What do you base this statement on?
Can you point to one pro who learned Go and got strong by following your method rather than the traditional approach? Can you point out any strong ama who did the same?

From where I stand, most Go books still teach the traditional method, and most go players still learn using it as well.
Do you have information to the contrary?

[oren]

Uh, we had that discussion in 1997. Since then, things have changed to a shrinking fraction of traditional and a growing fraction of reason-orientated methods. I am not surprised.

Evidence of this? This seems to be even more ridiculous than your usual statements.

Unless the shrinking fraction is a couple people you've been teaching with your methods, no one else I've seen does it in quite your way.

[Bartleby]

The fact that I look at other similar fields. For example: chess. I know a lot of formal research has been done in chess, especially in the old Soviet Union, tremendous amounts of money have been spent, and yet it did not seem to have contributed very much.

People refusing to apply it stay weak, while programs willing to apply it have beaten even top chess players;)

it seems that the best and fastest way to learn chess is still the old-fashioned way of rote memorization, examples, tactical problem solving, and simply experience.

It is a myth you and others like to cultivate, but it is fundamentally flawed. Learning by examples alone never causes great strength improvement. It is always required to be accompanied by understanding, so that the learning by examples (even if done subconsciously) knows what to calibrate for. No non-go player without any go knowledge besides the rules becomes strong by simply watching positional patterns of examples, quite like nobody discovers he laws of photosynthesis by looking the shapes of leaves and nobody becomes rich by appreciating the art on banknotes and coins.

Really, this is such nonsense. No one gets good at a intellectual board game just by learning theory. Playing such a game is an applied science, not a theoretical one. That's precisely why applying your knowledge repeatedly by doing problems/exercises is the most efficient way to get better at these games.


Robert, your comments about chess reveal ignorance about why computers are stronger at chess than humans, and the way strong chess players think. Pattern recognition is 80% of everything in chess, and that pattern recognition is mostly trained by solving practical problems and checking the results. Theory makes up only a part of the remaining 20%.

[John Fairbairn]

http://www.chessvibes.com/review-best-play

For the 1.543 people interested in this topic, thevabove review reveals remarkable similarities with the present debate.

[hyperpape]

It strikes me that the current work on go programs is not formal in the sense that Robert does formal work. I am not sure there is any mutual relevance.

[RobertJasiek]

What do you base this statement on?

More formal research. More, and an increasing percentage of literature, with methodical contents. More, and an increasing percentage of, teaching with methodical approach.

Can you point to one pro who learned Go and got strong by following your method rather than the traditional approach?

Not "my" method, but in general "methods relying on systematic approach or reasoning". I have not watched pros during their learning; I can only infer from their current statements or teaching. Example of a pro with strong indication of non-traditional statements: Jeong.

Can you point out any strong ama who did the same?

I am relying on related statements by Saijo: Catalin. (Of course, not the same; there are few doing formal research by themselves.)

From where I stand, most Go books still teach the traditional method,

The most unfortunately. Luckily, for Western books, the relative situation is better. E.g. Yang and Bozulich have changed from teaching by examples only to also teaching theory and principles.

and most go players still learn using it as well.

Asian players: presumably. They have less chance for learning non-traditionally, because the Asian literature and Asian teachers are changing more slowly WRT their relative percentages of teaching methods.

Western players: roughly 50% mainly learning by examples, 50% learning by reasoning, methods, principles and examples.

Do you have information to the contrary?

Yes. The most reliable such information I have is the regular acceptance of knowledge input not only by examples, but also a lot of reasoning, methods, principles, from ca. 95+% of my pupils.

no one else I've seen does it in quite your way.

Read, e.g., Tesuji (Davies), Fundamental Principles and Counting Liberties, Endgame (Ogawa), All About Ko, Mathematical Go Endgames, and open your eyes. Not "quite" my way, of course, but every author has his preferences:)

No one gets good at a intellectual board game just by learning theory.

And no one has said so. E.g., I have said many times to learn by theory, examples, reviewing AND application etc.

That's precisely why applying your knowledge repeatedly by doing problems/exercises is the most efficient way to get better at these games.

Whether it is the most efficiently depends on the learner! (That is why I provide examples together with offered theoretical knowledge, and the number of diagrams per page on average in my texts is easily comparable to teaching by examples only texts.)

Theory makes up only a part of the remaining 20%.

Would the chess programs' pattern matching work without the theory?

[Bantari]

What do you base this statement on?

More formal research. More, and an increasing percentage of literature, with methodical contents. More, and an increasing percentage of, teaching with methodical approach.

The we might diverge here on our formal definition of what "your method" means. I knew we stumble upon formality somewhere in our discussion, was bound to happen sooner or later.

To clarify: The books you cite in this post, and many others, do apply a more 'methodical' approach to presenting their content, but from what I understand there were always books which were more methodical and less methodical, for as long as books were written. Now we have more and more books translated, so no wonder there is more and more methodical books translated as well. And no wonder that westerners, like Davies, organize their books slightly more methodically.

But 'methodical' is not what I would say to describe what I call 'Your Method' (or YM for short) although methodical approach is certainly a part of it.
The differentiator is not being methodical, but presenting the knowledge in terms of problems and examples and short explanations vs. presenting it in form of formal definitions, theorems, and conclusions. Of course, both types of books need examples and problems, just like modern calculus books (which I take as a decent examples of YM) also have examples and problems. But these auxiliary only, the message is delivered and conveyed by more formalized means.

Traditional Approach or Common Approach (or CA for short) - is organized the other way around - the message/knowledge is conveyed by a series of examples, with often minimal explanation, and backed up by problems. Sometimes, like in problem books, the examples are minimal in scope themselves.

Both kinds of books/teaching can be more and/or less methodical, this by itself does not change the method used.

In this light, books you cite in this post, Davies and so on, are all using the traditional method, and other than being slightly more methodically organized than some other books, have not much to do with what I see as YM. Its all CA from where I stand.