It is currently Thu Mar 28, 2024 2:14 pm

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 22 posts ]  Go to page 1, 2  Next
Author Message
Offline
 Post subject: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #1 Posted: Tue Feb 26, 2019 2:11 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
I've finally got round to making a video about how to use Lizzie to review games. This is just a 1st draft, I plan to make a slicker version for the London Go Centre's Youtube channel and would welcome feedback on what you think I could improve. I think I got rather carried away with maybe too much Go content rather than focused on Lizzie instruction and ended up going for over an hour, but then discovered Quicktime only lets you upload 15 minutes even if your Youtube account allows longer as mine does. So here are 2 videos for the first 30 minutes. What do you think?




Here's my thoughts so far:
- Sometimes I waffle too much, e.g. should just say "you can use Lizzie to review games", no need to say your games, other games, pro games, joseki etc.
- Will add a more focused introduction when showing Lizzie, about the sort of questions it can answer and lack of wordy explanations.
- How long overall do you want the video? More or less LZ/Lizzie intro? I could put a table of contents with time links to skip to the sections.
- Did I address enough the how kyu players can use it well? Maybe I should review a 10k game rather than 4d game as example? Do weaker players end up thinking "Well yeah great, you can use Lizzie because you are 4 dan so can interpret its output, but I'm lost"? I want to help make these tools widely accessible and useful.
- Audio was using MacBook internal mic and generally ok but sometimes some hiss, maybe from when LZ was using processor.
- Headshot video was unnecessarily wide aspect ratio but I couldn't find a way to crop it in cheap-and-cheerful Quicktime.


This post by Uberdude was liked by 4 people: apetresc, sorin, thirdfogie, xela
Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #2 Posted: Tue Feb 26, 2019 3:38 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
Here are a few quick thoughts on parts 1 and 2:

1) The intro is too long. The information that I would find useful is:
  • What is LeelaZero/Lizzie, where to download it.
  • How strong is LZ with a CPU? With a medium GPU? Which networks should be used on which hardware?
  • Where to find alternative networks (Elf, Leela Master, 157a, etc.)?

2) It would be useful to add subtitles indicating which main idea you are trying to illustrate. Sometimes we can get lost, trying to follow Go ideas and at the same time trying to understand how to use Lizzie.

3) 4d vs. kyu games: you showed that some of LZ's blue moves are not so great once you play out the variation and show her the moves she didn't think about. That's useful information for players of all levels. Another advice that is useful for kyu players is to playout variations, but only choose among moves that look "natural" and simple enough.

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #3 Posted: Tue Feb 26, 2019 6:07 am 
Beginner

Posts: 15
Liked others: 4
Was liked: 3
I think you should make it into several videos based on the topics you cover since different people may be interested in different ones.

Also the second video had some background noise which it would be nice if you could remove.

Anyways I think the content you try to cover is very good, and it's nice to see someone taking the time to explain all that stuff.

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #4 Posted: Tue Feb 26, 2019 10:41 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Uberdude wrote:
I've finally got round to making a video about how to use Lizzie to review games.


Many thanks. :D This is an important topic, but one which is daunting to many players, both in terms of computer technique and in terms of improving one's go. You are providing a real service to us all. :)

Quote:
I think I got rather carried away with maybe too much Go content rather than focused on Lizzie instruction and ended up going for over an hour, but then discovered Quicktime only lets you upload 15 minutes even if your Youtube account allows longer as mine does. So here are 2 videos for the first 30 minutes. What do you think?


I agree with the sentiment that it would be good to break up the presentation into a number of short videos. Some software that I am learning has a series of tutorials, each of which addresses one point and is 3 min. long. IMO, that's too short, but 5 min. might be good. 15 min. is long enough for someone who is not generally familiar with the material to get lost.

Quote:
Here's my thoughts so far:
- Sometimes I waffle too much, e.g. should just say "you can use Lizzie to review games", no need to say your games, other games, pro games, joseki etc.


No, that's OK. There is a mnemonic for giving a talk: Say A Few Words. That is, State your idea, Accentuate your idea, For instance your idea, Wind up with your idea. Offering examples is fine. OC, if you don't think that the idea is worth emphasizing, then don't give examples. But I think that you should trust your instincts. You felt like giving examples, and for what you were trying to say, you were probably right. :)

Quote:
- I could put a table of contents with time links to skip to the sections.


Excellent! Different people will have different questions at different times. Letting them pick and choose is a good idea.:)

Quote:
- Did I address enough the how kyu players can use it well? Maybe I should review a 10k game rather than 4d game as example? Do weaker players end up thinking "Well yeah great, you can use Lizzie because you are 4 dan so can interpret its output, but I'm lost"? I want to help make these tools widely accessible and useful.


That's a big question, and an important one. I think that you need feedback from kyu players about that. I know that a number of top bot self play games have left me feeling bewildered. My response was like that when I was a child learning bridge. Wow! What happens next? Not trying to understand, but simply grokking it. But not everybody can take that approach or feels that it is useful. People need a guide through the labyrinth.

One thought that I have had is for players to use a version of LZ that is not too much stronger than they are, with the knowledge that sometimes it will suggest bad moves. That way they may be able to stretch their knowledge and understanding without getting overwhelmed. And, with the next higher level version available, they can move on when they are ready. Gradus ad parnassum. :)

Another thought is to have a video or two with Q&A with a kyu player or two. That format is a pretty good one. :)

----

This may just be me, but I do not find those long variations helpful, especially when they are presented instantaneously. I know that the deeper you go, the more questionable they become. Sometimes they give a big picture, but often my response is Huh? If I were preparing a presentation using variations, I would have LZ explore each move. You don't have to go very deep to discover that it will recommend a different play than you see in the instant variation that it shows. What's the point of showing several moves that go down the "wrong" path from there?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #5 Posted: Tue Feb 26, 2019 12:28 pm 
Gosei

Posts: 1733
Location: Earth
Liked others: 621
Was liked: 310
1.) Consider to not use a facecam.

(You are not streaming, but producing a tutorial video. In my opinion a facecam does not add to this kind of video.)


2.) Have a look at a better microphone, for example:

https://www.thomann.de/gb/the_t.bone_sc_420_usb_desktop_set.htm?ref=search_prv_1
(These are not too expensive and improve sound quality a lot. Be fast before brexit if you like this one ;-))


Last edited by Gomoto on Tue Feb 26, 2019 4:20 pm, edited 6 times in total.
Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #6 Posted: Tue Feb 26, 2019 12:53 pm 
Lives in sente

Posts: 923
Location: UK
Liked others: 72
Was liked: 479
Rank: 5 dan
KGS: macelee
This is great. As already suggested by others, it would be nice to break the material into several shorter videos - around 5 minutes each - this is what's done in professional training platforms. I guess nowadays most people are just too busy to dedicate 15 minutes to watch one video.


This post by macelee was liked by 2 people: Gomoto, jptavan
Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #7 Posted: Tue Feb 26, 2019 1:04 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
macelee wrote:
This is great. As already suggested by others, it would be nice to break the material into several shorter videos - around 5 minutes each - this is what's done in professional training platforms. I guess nowadays most people are just too busy to dedicate 15 minutes to watch one video.


I have noticed many popular chess videos are 15 - 25 minutes long to review a single game. To me that suggests that 30 - 45 minutes would be good to review a single go game. BTW, Ezra Pound thought that the French achieved intellectual superiority in Europe because their universities reduced lecture times to 45 minutes from one hour. ;)

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #8 Posted: Tue Feb 26, 2019 1:32 pm 
Lives in gote

Posts: 388
Liked others: 416
Was liked: 198
Thank you for doing this!

Any plans to make an equivalent written guide, with screenshots?
Sometimes information is easier to digest in written form.

_________________
Sorin - 361points.com

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #9 Posted: Tue Feb 26, 2019 1:45 pm 
Oza

Posts: 3647
Liked others: 20
Was liked: 4626
Quote:
Ezra Pound thought that the French achieved intellectual superiority in Europe because their universities reduced lecture times to 45 minutes from one hour. ;)


Ezra Pound also thought he could translate ancient Chinese and Japanese poetry without having learnt either the modern or the ancient languages. But, I suppose as he might have said, "Imagine that!"

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #10 Posted: Tue Feb 26, 2019 1:54 pm 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
John Fairbairn wrote:
Quote:
Ezra Pound thought that the French achieved intellectual superiority in Europe because their universities reduced lecture times to 45 minutes from one hour. ;)


Ezra Pound also thought he could translate ancient Chinese and Japanese poetry without having learnt either the modern or the ancient languages. But, I suppose as he might have said, "Imagine that!"


A true genius! ;)

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #11 Posted: Tue Feb 26, 2019 10:54 pm 
Lives in gote

Posts: 388
Liked others: 416
Was liked: 198
Gomoto wrote:
1.) Consider to not use a facecam.
(You are not streaming, but producing a tutorial video. In my opinion a facecam does not add to this kind of video.)


I actually liked the face camera, since it adds a nice personal touch to the tutorial.

The videos are very long though, they can be made a lot shorter for people with short attention spans like myself :-)

_________________
Sorin - 361points.com


This post by sorin was liked by 2 people: ez4u, Gomoto
Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #12 Posted: Wed Feb 27, 2019 2:45 am 
Oza

Posts: 3647
Liked others: 20
Was liked: 4626
Andrew

If cost (e.g. hardware, software, collaboration) is an aspect that affects what you can do, do remember you can apply to the TMH Fund. So long as you can show it's boosting the LGC specifically, there is no obvious reason why you shouldn't get serious consideration. I speak as one of the directors.

And if you add subtitles and keep the volume up, you are more likely to get my vote :)

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #13 Posted: Wed Feb 27, 2019 3:23 am 
Dies with sente

Posts: 93
Liked others: 26
Was liked: 13
Rank: Total beginner
Universal go server handle: Satorian
First of all, you doing lecture videos is highly appreciated! Given that you are a very strong player and that you spend a lot of time around the AI topics, your videos on Go in general have me very interested.

As for the technical aspects of the video: Yes, of course, they could be made nicer and more glossy, but it's actually fine. Personally, as a DDK, your explanations during the game discussion lost me as it was too fast and cursory for me to comprehend. I would need a different pace and see more moves being played out manually instead of having the variation pop up.

Some questions and issues that I find interesting that you might cover during future videos:

Which network and parameters are the best to learn from at different strength levels?
How would I use Lizzie or GRP or other tools to have a match analyzed, show me my 3-5 biggest blunders, and then show one or two viable variations per blunder up to a reasoable depth how the game could have progressed, in a way that I can learn from?
What are AI trends weaker players can or should adapt, other from those they shouldn't because they most likely can't follow through on the moves and concepts.

I terms of running time I agree that 45 minutes is a good length to aim for, with wiggle room for up to 60 minutes if a topic needs to be closed properly. That should allow some depth without becoming exhaustive.

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #14 Posted: Wed Feb 27, 2019 6:22 am 
Lives with ko

Posts: 128
Liked others: 148
Was liked: 29
Rank: British 3 kyu
KGS: thirdfogie
On the general question of "can a kyu player benefit from Leela Zero?", here are my thoughts as a British 4kyu (on a good day).

I run some of my games through Leela using Lizzie as the interface. The focus is on the following topics.

  • Moves where Leela thinks I lost at least 10% winning percentage.
  • Situations where there were difficult choices at the time of playing - can I win this fight, and if so, how?
  • I regularly take 9 stones against Matthew Mcfadyen and lose more often than I win. It is interesting to replay those games to find out exactly where the winning percentage changed sides. It is often much later than I thought at the time, so I have sometimes been giving up too soon.

In the first two cases, Leela often suggests sequences that I would never be able to play, and nor would an SDK opponent. But equally often it suggests an obviously better move that is within my Go vocabulary, but which I just didn't think of at the time. That is helpful.

At my level, there is no need to worry about having the latest network or achieving 100,000 playouts.

Now I'll view Uberdude's videos. And I also thank him for his work.


Last edited by thirdfogie on Wed Feb 27, 2019 8:14 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #15 Posted: Wed Feb 27, 2019 7:18 am 
Oza
User avatar

Posts: 2408
Location: Ghent, Belgium
Liked others: 359
Was liked: 1019
Rank: KGS 2d OGS 1d Fox 4d
KGS: Artevelde
OGS: Knotwilg
Online playing schedule: UTC 18:00 - 22:00
Something that hasn't been addressed by others: for a non-native speaker, your accent may be a tad strong. We have become accustomed to the American English style so that British English and a fortiori its local variants, require a higher command of the language to anticipate on the words you're saying.

Myself I am a pretty lazy adopter of technology and I found Lizzie very easy and intuitive to use. I don't need to know what LeelaZero or Lizzie is, its development cycles or even the basic "how to".

What a person like me needs is:
- a stronger player talking about the way to interpret LZ's suggestions and explain its pitfalls
- the whole "weights" thing, how to configure LZ at different levels of strength
- whether my computer is developing a local AI while using LZ, or is my LZ part of a whole AI that develops itself while I'm using it, or nothing of the kind?
- and should I regularly download a new version, not only for feature extensions but more importantly for AI strength?

As for the aspects already discussed:
- face is ok; dwyrin has the best balance and technique to have both face and board
- 1 video with linked chapters is ok, but indeed having multiple videos is better

Thumbs up for the effort!

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #16 Posted: Wed Feb 27, 2019 7:32 am 
Gosei
User avatar

Posts: 1753
Liked others: 177
Was liked: 491
Knotwilg wrote:
Something that hasn't been addressed by others: for a non-native speaker, your accent may be a tad strong.


I am also a non-native speaker who is more used to american English but had no difficulties with Uberdude's accent. On the other hand, I can imagine that the pace may be a bit fast for many non-native speakers, so if the video is meant to be for an international audience, it could be useful to
  • either slow down a bit
  • and/or add a few subtitles (no need to reproduce every sentence, but only key ideas).

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #17 Posted: Wed Feb 27, 2019 7:39 am 
Oza
User avatar

Posts: 2408
Location: Ghent, Belgium
Liked others: 359
Was liked: 1019
Rank: KGS 2d OGS 1d Fox 4d
KGS: Artevelde
OGS: Knotwilg
Online playing schedule: UTC 18:00 - 22:00
thirdfogie wrote:
In the first two cases, Leela often suggests sequences that I would never be able to play, and nor would an SDK opponent. But equally often it suggests an obviously better move that is within my Go vocabulary, but which I just didn't think of at the time. That is helpful.


That is interesting! In my case (1-2d) it is different.

I would say that LZ's suggestions are remarkably often moves that are in my vocab and much less often the kind of "ear reddening moves" I would expect AI to come up with. Most of my "blind spots" are not moves I'm unable to see but moves I dismiss because I misread what's underneath it or because I have "learned" they are bad (the early 3-3 is the prime example of that, for all of us, and has made it back into our repertoire).

The most instructive lessons from LZ is its relentless quest for sente, playing elsewhere asap and leaving stones or groups in a less stable state than we have been taught, like without a base. As said elsewhere, I have also learnt that attacking should either bring immediate profit in the form of territory, or build a framework with an existing group at the other side.

On a more general level, I'm trying to acquire LZ's language and "see sequences" from frequent analyses. While practicing this, I have realized I see many candidate moves but tend to evaluate those at face value or even convince myself my first choice is the right one, instead of seeing the sequence behind it.

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #18 Posted: Thu Feb 28, 2019 7:07 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Thanks all for the feedback. I'm busy the next few days but hopefully next week I'll have time for another go.

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #19 Posted: Thu Feb 28, 2019 11:17 am 
Lives with ko
User avatar

Posts: 205
Liked others: 49
Was liked: 36
Rank: EGF 2k
KGS: MKyle
I watched this out of curiosity, thinking that it wasn't really for me. I got the impression from the title of the thread and videos that this was an install and basic use guide for people brand new to lizzie. I was quite pleased with the go content though - in terms of practically interpreting bot advice.

I agree with what's been said about different videos for different sections. Kyu games would be good, but also I'd be interested to see the pro game review too

Top
 Profile  
 
Offline
 Post subject: Re: Video: Reviewing with LeelaZero/Lizzie (1st draft)
Post #20 Posted: Fri Mar 01, 2019 5:59 am 
Beginner

Posts: 16
Liked others: 11
Was liked: 2
Rank: DDK
KGS: 11 kyu
IGS: 14 kyu
OGS: 11 kyu
Universal go server handle: jonsa
Uberdude, I thought your content and delivery was good; I appreciated that you weren't reading from a script (as some tutorials do -- they sound very stiff). It sounded to me like you have a very good grasp of Lizzie/LeelaZero AND are able to convey that (in addition to your Go knowledge). Others have mentioned things that could be polished, but on the whole I think it is very good. I am 10 kyu and thought your comments on a 4D game were perfectly intelligible.

The only thing I would comment on is that I still do not know how to install Lizzie on a Mac. I get the impression that you are pretty comfortable with computers (as you have installed it :D ) and, likewise, that many on this forum are also computer competent. I am not: I open the Lizzie read-me and I don't understand what to do with 'example of compiling Leela Zero'. I do not know what config.h is or how to open it in order to compile the CPU-only version of Leela Zero. I read on an old r/cbaduk post that one must be comfortable with cmake/make. I do not know what these things are.

What would be helpful for me (I hope that I am not alone in this) is a tutorial where someone went step by step: downloading lizzie, downloading the programs (or at least mentioning them; you mentioned Java, which was helpful) needed to compile Lizzie, and then compiling it.

This sort of tutorial I desire may not be your goal, but I haven't seen anybody with my problem here, so I thought I would take the chance to ask. Regarding using Lizzie and reviewing with it, the video is excellent.


Edit: I just found a recent post on cbaduk 'troubleshooting Leela Zero on Mac' which I think answers my plight (so I have taken a large chunk of my post and hidden it).

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 22 posts ]  Go to page 1, 2  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group