It is currently Tue Apr 16, 2024 6:19 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next
Author Message
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #1 Posted: Wed Sep 13, 2017 3:13 pm 
Lives with ko

Posts: 136
Liked others: 47
Was liked: 21
Rank: KGS 6 dan
I 've been having the exact same issue on my average laptop.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #2 Posted: Wed Sep 13, 2017 3:49 pm 
Lives in sente

Posts: 902
Location: Fort Collins, CO
Liked others: 319
Was liked: 287
Rank: AGA 3k
Universal go server handle: jeromie
I haven't tried running it, but I wonder if it's this item from the change log:

Quote:
New, much stronger Monte Carlo evaluation by combining Policy Gradient Reinforcement Learning and Simulation Balancing.


That seems consistent with extra time being in the evaluation process.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #3 Posted: Thu Sep 14, 2017 4:45 am 
Beginner

Posts: 3
Liked others: 0
Was liked: 0
is twogtp.py available opensource? I can post results for comparison from a machine with a large gpu and linux.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #4 Posted: Thu Sep 14, 2017 9:25 am 
Dies with sente

Posts: 93
Liked others: 26
Was liked: 13
Rank: Total beginner
Universal go server handle: Satorian
Given the release notes, it sounds like the playouts are a lot heavier than before, with a lot more attached to them. Playout count may go down, but overall level and playing quality should increase.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #5 Posted: Fri Sep 15, 2017 12:46 pm 
Dies in gote

Posts: 34
Liked others: 0
Was liked: 10
Satorian wrote:
Given the release notes, it sounds like the playouts are a lot heavier than before, with a lot more attached to them. Playout count may go down, but overall level and playing quality should increase.


From the same release notes:
"More accurate, larger value network for 19x19."

Larger network would also take longer to compute.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #6 Posted: Sat Sep 16, 2017 7:48 pm 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
This person said that Leela 11 beta is 2 stones stronger than Leela 10

https://www.reddit.com/r/baduk/comments ... nger_than/



By the way, Beta 5 is out, the changelog is

Windows version is now compiled with Clang/LLVM 5.0 instead of MSVC2017. This makes the Monte Carlo evaluations about 15% faster.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #7 Posted: Tue Sep 19, 2017 8:29 am 
Beginner

Posts: 3
Liked others: 0
Was liked: 0
I started on a 40 game series but 0.11 won the first 12 of them so I'll restart with 0.5 komi to get into 0-100% range.

Different from above, I used 7.5 komi as leela plays by chinese rules.

command:
Code:
~/pachi/tools/twogtp.py --black '0.10.0/leela_0100_linux_x64_opencl --gtp --quiet --noponder' --white '0.11.0b5/leela_0110_linux_x64_opencl --gtp --quiet --noponder' --komi 7.5 --games 40


0.10 openCL benchmarks:
Code:
Leela: benchmark
200000 games in  7.55 seconds -> 26490 g/s (3311 g/s per thread)
Avg Len: 451.69 Score: -1.961805

Leela: netbench
2000 predictions in  1.45 seconds -> 1379 p/s
10000 evaluations in  3.44 seconds -> 2906 p/s


0.11b5 openCL benchmarks:
Code:
Leela: benchmark
200000 games in 10.76 seconds -> 18587 g/s (2323 g/s per thread)
Avg Len: 418.58 Score: -2.588590

Leela: netbench
2000 predictions in  3.11 seconds -> 643 p/s
10000 evaluations in  3.40 seconds -> 2941 p/s


Looking at benchmark results predictions are slower(explained by larger value net?) while evaluations are roughly equal(same sized policy net?).

As for playing strength / game quality, having a GPU seems almost a must.

First 5 games attached below:









Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #8 Posted: Wed Sep 20, 2017 8:41 am 
Beginner

Posts: 3
Liked others: 0
Was liked: 0
so far 0.11b5 as white has won 7 out of 8 games on 0.5 komi

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #9 Posted: Wed Sep 20, 2017 9:57 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
beta 9 is out and changelog didn't mention any strength-related improvement
Judging by the number, if nothing wrong happen the completed version 11 will release soon.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #10 Posted: Mon Sep 25, 2017 1:14 pm 
Lives in gote

Posts: 436
Liked others: 1
Was liked: 38
Rank: KGS 5 kyu
I am just a mid kyu level player but holy detritus is Leela strong.
Bot easily gives me 6 handicap and I struggle.
I can't imagine what AlphaGo level is.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #11 Posted: Mon Sep 25, 2017 3:38 pm 
Lives with ko

Posts: 136
Liked others: 47
Was liked: 21
Rank: KGS 6 dan
Krama wrote:
I am just a mid kyu level player but holy detritus is Leela strong.
Bot easily gives me 6 handicap and I struggle.
I can't imagine what AlphaGo level is.

I feel you. Too strong... even now (Leela 0.11 beta 10)...

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #12 Posted: Tue Sep 26, 2017 5:52 am 
Beginner

Posts: 2
Location: Vietnam
Liked others: 5
Was liked: 0
I think that Leela is stronger is the most important, slower is not a problem.

_________________
http://toannguyenthanh.rf.gd/


Last edited by toannguyenthanh on Sun Oct 22, 2017 2:15 am, edited 1 time in total.
Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #13 Posted: Tue Sep 26, 2017 11:51 am 
Lives in gote

Posts: 436
Liked others: 1
Was liked: 38
Rank: KGS 5 kyu
johnsmith wrote:
Krama wrote:
I am just a mid kyu level player but holy detritus is Leela strong.
Bot easily gives me 6 handicap and I struggle.
I can't imagine what AlphaGo level is.

I feel you. Too strong... even now (Leela 0.11 beta 10)...


You are a 6 dan on kgs? So that is like 4 dan EGF right?

How well can you fare against gpu version on unlimited simulations (let's say 30 min)

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #14 Posted: Wed Sep 27, 2017 1:43 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Krama wrote:
You are a 6 dan on kgs? So that is like 4 dan EGF right?

As an aside, I don't think many 4d EGF are 6d KGS, I'm certainly not. There are 6d EGF who are 6d KGS like Lukas Kraemer, or Cornel Burzo was 6d for a long time and just got back to 7d. KGS dan ranks are pretty tough these days. Ilya and Pavol (7d/1p) are only 7d KGS now.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #15 Posted: Wed Sep 27, 2017 3:59 am 
Lives with ko

Posts: 136
Liked others: 47
Was liked: 21
Rank: KGS 6 dan
Krama wrote:
You are a 6 dan on kgs? So that is like 4 dan EGF right?
How well can you fare against gpu version on unlimited simulations (let's say 30 min)


Well, I don't play on KGS, only on Tygem. And I'm 7d over there. I guess that's something like solid 5d EGF :)

Anyways, LEELA(B) on Tygem has 50:50 score against pros. That surely tells something about its strength. And it only plays against pros.
Personally, I like to play with RoyalLeela on OGS (thx to roy777 who has Ryzen 1700 CPU and 1080 Ti GPU), and I have 50:50 against Leela on 3 handicap stones; winrate on 4 handi is 100% though I struggle a little bit :) In even games RoyalLeela just smashes. Like Hulk.
On my personal average laptop, in even games I don't stand a chance either, though I have 100% score against 3 handi. And so far I won 2 games in Sunjung Baduk.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #16 Posted: Wed Sep 27, 2017 4:05 am 
Lives in gote

Posts: 418
Liked others: 9
Was liked: 83
Rank: kgs 5 kyu
KGS: Pio2001
One of our club members is 7d on Tygem, but 1 kyu FFG (EGF).

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #17 Posted: Wed Sep 27, 2017 4:23 am 
Judan

Posts: 6725
Location: Cambridge, UK
Liked others: 436
Was liked: 3719
Rank: UK 4 dan
KGS: Uberdude 4d
OGS: Uberdude 7d
Pio2001 wrote:
One of our club members is 7d on Tygem, but 1 kyu FFG (EGF).

France is notorious for its under-ranked players (particularly young improvers), but also FFG rank != EGF rank, they have their own parallel internal rating system as well as submitting to EGD. Is this player 1 kyu in the EGF system, and has he gone about 50/50 in a tournament, such as the EGC, not just full of under-ranked French? I could believe a solid EGF 2d good at blitz is Tygem 7d, but not a 1k.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #18 Posted: Wed Sep 27, 2017 5:45 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
johnsmith wrote:
Anyways, LEELA(B) on Tygem has 50:50 score against pros. That surely tells something about its strength. And it only plays against pros.

Very impressive. I also notice that Leela has 50% winrate against Fox 9d. Can we unanimously declare that Leela is the strongest freeware go AI? (and strongest go AI available to public right now, just before Zenith Go 7 release in upcoming months)

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #19 Posted: Wed Sep 27, 2017 6:01 am 
Lives with ko

Posts: 136
Liked others: 47
Was liked: 21
Rank: KGS 6 dan
pookpooi wrote:
johnsmith wrote:
Anyways, LEELA(B) on Tygem has 50:50 score against pros. That surely tells something about its strength. And it only plays against pros.

Very impressive. I also notice that Leela has 50% winrate against Fox 9d. Can we unanimously declare that Leela is the strongest freeware go AI? (and strongest go AI available to public right now, just before Zenith Go 7 release in upcoming months)


What is Leela's name on Fox? I'd like to check out those games.
P.S. FineArt also started playing and is stronger than ever - only 1 loss and 35 wins. It has kinda new style, it prefers high short shimari it seems.

Top
 Profile  
 
Offline
 Post subject: Re: Leela 0.11.0 Beta version slower than 0.10.0 ?
Post #20 Posted: Wed Sep 27, 2017 6:47 am 
Lives in sente

Posts: 727
Liked others: 44
Was liked: 218
GD Posts: 10
Leela测试

And yes, FineArt, DeepZenGo, CGI strength are another league, they're all 10d fox (with Ke Jie join as the only 10d human).
FineArt is able to maintain more than 60% winrate even giving 2-4 stones handicap to 9d. And to think that AlphaGo is likely to be 3 stones stronger than them is insane.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group