Engine Tournament

Vargo · **#241**

as0770 wrote:

If one engine scores 55% in a 400 game match, there is still a chance of 1% that it is _not_ stronger.

It's even more, around 2.7%

q30 · **#242**

as0770 wrote:

Or a weight file of similar strength. And once again: If one engine scores 55% in a 400 game match, there is still a chance of 1% that it is _not_ stronger.

I didn't argue, that v. 0.14 absolutely stronger than v. 0.15, but most likely it's not weaker. And IMHO if the next version not stronger, than previous, that it's weakness of first one...

as0770 wrote:

Your link points to LeelaZero_PhoenixGo v1 and v2 but you were talking about Leela Zero v0.14 and 0.15. LeelaZero 0.14 don't support Phoenix networks. Results of LeelaZero_PhoenixGo v1 vs. LeelaZero_PhoenixGo v2 with Phoeniox Networks don't say anything about the strength of LeelaZero v0.14 and 0.15. I wouldn't even trust results with Leela networks.

If You'll open really page by the link, You'll find the next text (content of Readme.md file):

Quote:

So 'v1' & 'v2' are only the folders (i.e. a parts of link paths) and the engines are: "leela-zero-0.14-phoenix-go" and "leela-zero-0.15-phoenix-go"...

q30 · **#243**

Vargo wrote:

as0770 wrote:

If one engine scores 55% in a 400 game match, there is still a chance of 1% that it is _not_ stronger.

It's even more, around 2.7%

And these quantitative calculations are true in used conditions (time control, number of playouts and so on) only...

as0770 · **#244**

q30 wrote:

I didn't argue, that v. 0.14 absolutely stronger than v. 0.15,

You did. Thats what all the discussion is about:

q30 wrote:

LeelaZero 0.14 is a bit stronger (not as much, as in case of Phoenix), than 0.15 (details)...

q30 wrote:

If You'll open really page by the link, You'll find the next text

Nevertheless it is not the same as the official Leela branch, did I already mention that Leela v0.14 doesn't support Phoenix networks?

q30 · **#245**

as0770 wrote:

q30 wrote:

I didn't argue, that v. 0.14 absolutely stronger than v. 0.15,

You did. Thats what all the discussion is about:

q30 wrote:

LeelaZero 0.14 is a bit stronger (not as much, as in case of Phoenix), than 0.15 (details)...

q30 wrote:

If You'll open really page by the link, You'll find the next text

Nevertheless it is not the same as the official Leela branch, did I already mention that Leela v0.14 doesn't support Phoenix networks?

I think, You agree, that 0.15 version is not stronger, than 0.14 one. For most usual users it's the same, that new version is weaker: it's no a reason to download it. Have You any result of sparring, where 0.15 version was at most a bit stronger,than 0.14 one?
You can do yourself 2 matches of v.014 vs v.0.15 with enough for You number of games with a very big time control (10 '/move, for example) and with GPU enabled: 1) with -p 1; 2) with -p 10000. If in second match will more % win of one engine, than it's stronger.

Version 0.14 of Phoenix LeelaZero supports Phoenix neuronets: see results

as0770 · **#246**

q30 wrote:

as0770 wrote:

q30 wrote:

I didn't argue, that v. 0.14 absolutely stronger than v. 0.15,

You did. Thats what all the discussion is about:

q30 wrote:

LeelaZero 0.14 is a bit stronger (not as much, as in case of Phoenix), than 0.15 (details)...

q30 wrote:

If You'll open really page by the link, You'll find the next text

Nevertheless it is not the same as the official Leela branch, did I already mention that Leela v0.14 doesn't support Phoenix networks?

I think, You agree, that 0.15 version is not stronger, than 0.14 one. For most usual users it's the same, that new version is weaker: it's no a reason to download it. Have You any result of sparring, where 0.15 version was at most a bit stronger,than 0.14 one?
You can do yourself 2 matches of v.014 vs v.0.15 with enough for You number of games with a very big time control (10 '/move, for example) and with GPU enabled: 1) with -p 1; 2) with -p 10000. If in second match will more % win of one engine, than it's stronger.

Version 0.14 of Phoenix LeelaZero supports Phoenix neuronets: see results

Don't you read the answers to your posts, do you just ignore them or don't you understand them? That is really ridiculous...

yakcyll · **#247**

I'm not a moderator here, but I'm pretty sure I'm not the only one to think this exchange has already gone far enough. If you want to keep throwing ad hominems around, it's best to keep it to private messages; pretty sure this thread was never meant to serve this purpose.

as0770 · **#248**

yakcyll wrote:

I'm not a moderator here, but I'm pretty sure I'm not the only one to think this exchange has already gone far enough. If you want to keep throwing ad hominems around, it's best to keep it to private messages; pretty sure this thread was never meant to serve this purpose.

I simply try to answer questions here. It is somehow frustrating when you take time and effort to explain simple little facts, and the respondent consequently ignores them. I don't know if he is just trolling or if he don't understands anything. The purpose of a public forum is to discuss in public. And btw your post is way more off topic than ours. If you don't like the thread, just ignore it.

q30 · **#249**

as0770 wrote:

...

Don't you read the answers to your posts, do you just ignore them or don't you understand them? That is really ridiculous...

I didn't understand English well. But the same question is for You...
If You yourself read and understand answers to Your posts, can You give answer to the question: You aren't agree, that Phoenix LeelaZero 0.14 is stronger, than 0.15 one (and if don't agree, than were is return proof in facts/results)?

Vargo · **#250**

Someone call the referee, plz.... ;-)

q30 · **#251**

The referee can be anyone with modern Nvidia video card, that will do the next:

Quote:

2 matches of v.014 vs v.0.15 with statistically enough number of games with a very big time control (10 '/move, for example) and with GPU enabled: 1) with -p 1; 2) with -p 10000. If in second match will more % win of one engine, than it's stronger (for more reliability one can add the third match with -p 1000 to receive intermediate value of % win).

(especially in case of Phoenix)...

as0770 · **#252**

q30 wrote:

You aren't agree, that Phoenix LeelaZero 0.14 is stronger, than 0.15 one

I didn't say it is not stronger. I said 50,8% winrate is far away from proofing anything, and you should simply accept this fact.

Especially when testing different versions of one engine, the small differences in strength will result in a higher winrate than when testing different engines. So for me 50,8% mean that both versions are equal in strength.

q30 wrote:

(and if don't agree, than were is return proof in facts/results)?

I don't want to proof it because I would have to play many thousands of games,

But why don't you take a look at Leela's changelog:

Code:

This is a bugfix release for training game generation.

    Bugfix to Dirichlet noise being more uniform than intended (bug was introduced in v0.13).
    Tweaks to randomized move selection in training games to reduce needless blunders. Added extra options.

https://github.com/gcp/leela-zero/releases

There is NO difference in gameplay for the official Leela and I don't care if there is a tiny difference in some Leela hack.

q30 · **#253**

How I had mentioned above, the difference in strength may be results from difference in performance, maden by changes in Phoenix LeelaZero code.
So, the only proof is "LeelaZero_phoenix_0.15 - LeelaZero_phoenix_0.14: 0 - 4", i.e. Phoenix LeelaZero v.0.14 is stronger, that v.0.15...

as0770 · **#254**

q30 wrote:

How I had mentioned above, the difference in strength may be results from difference in performance, maden by changes in Phoenix LeelaZero code.
So, the only proof is "LeelaZero_phoenix_0.15 - LeelaZero_phoenix_0.14: 0 - 4", i.e. Phoenix LeelaZero v.0.14 is stronger, that v.0.15...

If you roll a dice the chance of a 4-0 is 12,5%. If you play matches between two engines the chance of a 4-0 is >= 12,5% because possible duplicated moves, differences in compilation and of course differences in strength.

q30 · **#255**

Compilation was the same (PC, system soft and compilation parameters).
Because other proofs (with bigger %, i.e. in big plays number matches with GPU enabled) are absent, on this moment the approval "version 14 is stronger, that 15 one" is true...

as0770 · **#256**

q30 wrote:

Compilation was the same (PC, system soft and compilation parameters).
Because other proofs (with bigger %, i.e. in big plays number matches with GPU enabled) are absent, on this moment the approval "version 14 is stronger, that 15 one" is true...

You can't claim it to be true as long as it is not proven.

Talking about the result of 208:216... If both opponents have the same strength, the chance that one will score at least 216 points in a 424 games match is 73%.

q30 · **#257**

You have got the result 216:208 in favour of Phoenix LZ v.14?
And how many playouts there was in games beginning?

as0770 · **#258**

q30 wrote:

You have got the result 216:208 in favour of Phoenix LZ v.14?
And how many playouts there was in games beginning?

It's from the link you posted.

q30 · **#259**

I had not posted any link to any game results between Phoenix v.14 and v.15, except mine.
So, on this moment the only proof is, that Phoenix LZ v.14 is stronger, than v.15, with (from basics of probability theory) the probability of their equality (1/2)^4 and the probability of stronger v.15 <(1/2)^4, i.e. v.14 is stronger with >87,5%.

dfan · **#260**

If A and B are equally strong, A will beat B 4-0 with probability 1/16. This doesn't mean that if A beats B 4-0, the probability they are equally strong is 1/16.

Engine Tournament

Who is online