Engine Tournament

[Vargo]

If one engine scores 55% in a 400 game match, there is still a chance of 1% that it is _not_ stronger.

It's even more, around 2.7%

[q30]

Or a weight file of similar strength. And once again: If one engine scores 55% in a 400 game match, there is still a chance of 1% that it is _not_ stronger.

I didn't argue, that v. 0.14 absolutely stronger than v. 0.15, but most likely it's not weaker. And IMHO if the next version not stronger, than previous, that it's weakness of first one...

Your link points to LeelaZero_PhoenixGo v1 and v2 but you were talking about Leela Zero v0.14 and 0.15. LeelaZero 0.14 don't support Phoenix networks. Results of LeelaZero_PhoenixGo v1 vs. LeelaZero_PhoenixGo v2 with Phoeniox Networks don't say anything about the strength of LeelaZero v0.14 and 0.15. I wouldn't even trust results with Leela networks.

If You'll open really page by the link, You'll find the next text (content of Readme.md file):

LeelaZero_PhoenixGo

LeelaZero + PhoenixGo's weights.

0.15 + lizzie
Source Code
https://github.com/yenw/LeelaZero_Phoen ... lizzie.zip
Weights
V1: https://github.com/yenw/LeelaZero_Phoen ... _v1.txt.gz
Leela Zero
Windows
CPU: https://github.com/yenw/LeelaZero_Phoen ... ie.exe.zip
GPU: https://github.com/yenw/LeelaZero_Phoen ... ie.exe.zip

0.15
Source Code
https://github.com/yenw/LeelaZero_Phoen ... nix-go.zip
Weights
V1: https://github.com/yenw/LeelaZero_Phoen ... _v1.txt.gz
Leela Zero
Windows
CPU: https://github.com/yenw/LeelaZero_Phoen ... pu.exe.zip
GPU: https://github.com/yenw/LeelaZero_Phoen ... cl.exe.zip

0.14
Source Code
https://github.com/yenw/LeelaZero_Phoen ... nix-go.zip
Weights
V1: https://github.com/yenw/LeelaZero_Phoen ... _v1.txt.gz
Leela Zero
Windows
CPU: https://github.com/yenw/LeelaZero_Phoen ... pu.exe.zip
GPU: https://github.com/yenw/LeelaZero_Phoen ... cl.exe.zip
Linux
CPU: https://github.com/yenw/LeelaZero_Phoen ... 64_cpu.zip
GPU: https://github.com/yenw/LeelaZero_Phoen ... opencl.zip
MacOS
CPU: https://github.com/yenw/LeelaZero_Phoen ... 64_cpu.zip
GPU: https://github.com/yenw/LeelaZero_Phoen ... opencl.zip

So 'v1' & 'v2' are only the folders (i.e. a parts of link paths) and the engines are: "leela-zero-0.14-phoenix-go" and "leela-zero-0.15-phoenix-go"...

[q30]

If one engine scores 55% in a 400 game match, there is still a chance of 1% that it is _not_ stronger.

It's even more, around 2.7%

And these quantitative calculations are true in used conditions (time control, number of playouts and so on) only...

[as0770]

I didn't argue, that v. 0.14 absolutely stronger than v. 0.15,

You did. Thats what all the discussion is about:

LeelaZero 0.14 is a bit stronger (not as much, as in case of Phoenix), than 0.15 (details)...

If You'll open really page by the link, You'll find the next text

Nevertheless it is not the same as the official Leela branch, did I already mention that Leela v0.14 doesn't support Phoenix networks?

[q30]

I didn't argue, that v. 0.14 absolutely stronger than v. 0.15,

You did. Thats what all the discussion is about:

LeelaZero 0.14 is a bit stronger (not as much, as in case of Phoenix), than 0.15 (details)...

If You'll open really page by the link, You'll find the next text

Nevertheless it is not the same as the official Leela branch, did I already mention that Leela v0.14 doesn't support Phoenix networks?

I think, You agree, that 0.15 version is not stronger, than 0.14 one. For most usual users it's the same, that new version is weaker: it's no a reason to download it. Have You any result of sparring, where 0.15 version was at most a bit stronger,than 0.14 one?
You can do yourself 2 matches of v.014 vs v.0.15 with enough for You number of games with a very big time control (10 '/move, for example) and with GPU enabled: 1) with -p 1; 2) with -p 10000. If in second match will more % win of one engine, than it's stronger.

Version 0.14 of Phoenix LeelaZero supports Phoenix neuronets: see results

[as0770]

I didn't argue, that v. 0.14 absolutely stronger than v. 0.15,

You did. Thats what all the discussion is about:

LeelaZero 0.14 is a bit stronger (not as much, as in case of Phoenix), than 0.15 (details)...

If You'll open really page by the link, You'll find the next text

Nevertheless it is not the same as the official Leela branch, did I already mention that Leela v0.14 doesn't support Phoenix networks?

I think, You agree, that 0.15 version is not stronger, than 0.14 one. For most usual users it's the same, that new version is weaker: it's no a reason to download it. Have You any result of sparring, where 0.15 version was at most a bit stronger,than 0.14 one?
You can do yourself 2 matches of v.014 vs v.0.15 with enough for You number of games with a very big time control (10 '/move, for example) and with GPU enabled: 1) with -p 1; 2) with -p 10000. If in second match will more % win of one engine, than it's stronger.

Version 0.14 of Phoenix LeelaZero supports Phoenix neuronets: see results

Don't you read the answers to your posts, do you just ignore them or don't you understand them? That is really ridiculous...

[yakcyll]

I'm not a moderator here, but I'm pretty sure I'm not the only one to think this exchange has already gone far enough. If you want to keep throwing ad hominems around, it's best to keep it to private messages; pretty sure this thread was never meant to serve this purpose.

[as0770]

I'm not a moderator here, but I'm pretty sure I'm not the only one to think this exchange has already gone far enough. If you want to keep throwing ad hominems around, it's best to keep it to private messages; pretty sure this thread was never meant to serve this purpose.

I simply try to answer questions here. It is somehow frustrating when you take time and effort to explain simple little facts, and the respondent consequently ignores them. I don't know if he is just trolling or if he don't understands anything. The purpose of a public forum is to discuss in public. And btw your post is way more off topic than ours. If you don't like the thread, just ignore it.

[q30]

...

Don't you read the answers to your posts, do you just ignore them or don't you understand them? That is really ridiculous...

I didn't understand English well. But the same question is for You...
If You yourself read and understand answers to Your posts, can You give answer to the question: You aren't agree, that Phoenix LeelaZero 0.14 is stronger, than 0.15 one (and if don't agree, than were is return proof in facts/results)?

[Vargo]

Someone call the referee, plz....

[q30]

The referee can be anyone with modern Nvidia video card, that will do the next:

2 matches of v.014 vs v.0.15 with statistically enough number of games with a very big time control (10 '/move, for example) and with GPU enabled: 1) with -p 1; 2) with -p 10000. If in second match will more % win of one engine, than it's stronger (for more reliability one can add the third match with -p 1000 to receive intermediate value of % win).

(especially in case of Phoenix)...

[as0770]

You aren't agree, that Phoenix LeelaZero 0.14 is stronger, than 0.15 one

I didn't say it is not stronger. I said 50,8% winrate is far away from proofing anything, and you should simply accept this fact.

Especially when testing different versions of one engine, the small differences in strength will result in a higher winrate than when testing different engines. So for me 50,8% mean that both versions are equal in strength.

(and if don't agree, than were is return proof in facts/results)?

I don't want to proof it because I would have to play many thousands of games,

But why don't you take a look at Leela's changelog:

Code: Select all

This is a bugfix release for training game generation.

    Bugfix to Dirichlet noise being more uniform than intended (bug was introduced in v0.13).
    Tweaks to randomized move selection in training games to reduce needless blunders. Added extra options.

https://github.com/gcp/leela-zero/releases

There is NO difference in gameplay for the official Leela and I don't care if there is a tiny difference in some Leela hack.

[q30]

How I had mentioned above, the difference in strength may be results from difference in performance, maden by changes in Phoenix LeelaZero code.
So, the only proof is "LeelaZero_phoenix_0.15 - LeelaZero_phoenix_0.14: 0 - 4", i.e. Phoenix LeelaZero v.0.14 is stronger, that v.0.15...

[as0770]

How I had mentioned above, the difference in strength may be results from difference in performance, maden by changes in Phoenix LeelaZero code.
So, the only proof is "LeelaZero_phoenix_0.15 - LeelaZero_phoenix_0.14: 0 - 4", i.e. Phoenix LeelaZero v.0.14 is stronger, that v.0.15...

If you roll a dice the chance of a 4-0 is 12,5%. If you play matches between two engines the chance of a 4-0 is >= 12,5% because possible duplicated moves, differences in compilation and of course differences in strength.

[q30]

Compilation was the same (PC, system soft and compilation parameters).
Because other proofs (with bigger %, i.e. in big plays number matches with GPU enabled) are absent, on this moment the approval "version 14 is stronger, that 15 one" is true...