It is currently Thu Mar 28, 2024 1:19 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 22 posts ]  Go to page 1, 2  Next
Author Message
Offline
 Post subject: Kombilo 0.8.1
Post #1 Posted: Wed Nov 02, 2016 1:31 pm 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
Kombilo is a free go database program with pattern search (in game collections and/or your own games, including search in variations), search for game information, references to commentaries in the English go literature, ... more information in the Kombilo documentation.

After a long break I was able to make some time for finishing version 0.8, with, among many small things, the following new features:
    Create SGF trees from an arbitrary starting position from your database. For instance, in this way you could make Fuseki or Joseki dictionaries.
    Custom menus (as in the ancient Kombilo 0.5) to add short-cuts for pattern or game info searches which you use often.
    Translation to German, and the framework to add other languages. Please contact me if you would like to contribute to a translation to other languages

On Linux, Kombilo can be installed as a Python package via pip. On Windows, you can either install Python and install Kombilo via pip; or you can use the stand-alone Windows installer.

I did not have a chance to check installation on Mac OS X - in principle all the required components are available, so I would be hopeful that someone can make it work.

More information and download links (including links to the documentation, online or as pdf).

Feedback is appreciated - either in this thread, via email, or on github, where the source code is hosted.

Ulrich

_________________
u-go.net


This post by ugoertz was liked by 4 people: Bonobo, Drew, ez4u, jptavan
Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #2 Posted: Thu Nov 03, 2016 2:59 am 
Oza
User avatar

Posts: 2401
Location: Tokyo, Japan
Liked others: 2338
Was liked: 1332
Rank: Jp 6 dan
KGS: ez4u
Ulrich - Thanks for the update! A few issues right now though (on Windows 7; my database is just over 100K games).

1. The big one! Memory usage seems to be a problem. When I open 0.8.1 it occupies around 430 MB of memory (windows "working set" per task manager). If I search on the empty board, usage jumps immediately to 745 MB. If I keep searching, usage keeps climbing by 100+ MB per search. Finally after perhaps 12 or so searches, Windows intervenes...
Attachment:
Kombilo 8 out of memory warning 1 2016-11-03_18-14-10.jpg
Kombilo 8 out of memory warning 1 2016-11-03_18-14-10.jpg [ 492.02 KiB | Viewed 17396 times ]

At the point where this happened, task manager reported (note: I have only 4 GB of memory on my machine)...
Attachment:
Kombilo 8 out of memory warning 2 2016-11-03_18-14-55.jpg
Kombilo 8 out of memory warning 2 2016-11-03_18-14-55.jpg [ 417.75 KiB | Viewed 17396 times ]

In contrast, version 0.7.2 opens at about 427 MB and after the same set of searches rises to about 465 MB. In the past I have used it for hours at a time without any issues.

More like requests... :bow:
2. Please uncheck "accept duplicates" in the default installation of the "edit database list" dialog. When I tried to add my databases I did not notice this was turned on; kombilo automatically scanned and added all the previously excluded duplicates. Since I combine the GoGoD and Go4Go databases I have 40K+ duplicates. :-) I admit I am an extreme example but I imagine that most kombilo users would be working with duplicates turned off.

3. Please uncheck "Use upper-case labels" in the default installation of the "Edit advanced options" dialog. Alternatively (and maybe better) make the default output lower-case labels when the user chooses "wiki" as the Export mode under Export search results since upper-case are special markup in wiki (and L19) diagrams. Casual users (as opposed to hardcore :rambo: ) may be confused by this if they output diagrams and try to use them online.

4. Last (for now anyway!), could you change the type of widget used to select "Move limit" in the lower right corner Option dialog? It is very difficult to select exactly X moves with a mouse because the slider is too short for the number of moves (250). At times in fuseki searches I want to be specific (for example within exactly 7 moves). As my fingers have become less and less flexible with the years. I am less and less able to hit the number I want. A textbox that I could type the number in would be best.

As always though you have my heartfelt thanks for inventing kombilo in the first place and then for all your work over the years. :clap: :tmbup:

_________________
Dave Sigaty
"Short-lived are both the praiser and the praised, and rememberer and the remembered..."
- Marcus Aurelius; Meditations, VIII 21

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #3 Posted: Thu Nov 03, 2016 3:44 am 
Dies with sente

Posts: 99
Liked others: 5
Was liked: 33
ugoertz wrote:
Kombilo is a free go database program. Feedback is appreciated.

I started kombilo on a 100000-game tree, but it did not succeed importing that, it freezes halfway. (Some years ago I debugged this situation and found that kombilo would get stuck after opening all available file descriptors. This time I just observed that the kombilo process had 1024 fds open, maybe still the same problem.)

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #4 Posted: Thu Nov 03, 2016 3:54 pm 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
Dave,

thanks a lot for the detailed feedback!

ez4u wrote:
1. The big one! Memory usage seems to be a problem. When I open 0.8.1 it occupies around 430 MB of memory (windows "working set" per task manager). If I search on the empty board, usage jumps immediately to 745 MB.


That's obviously a serious problem. I think I have located it, but I need to check this a little more thoroughly. (It is much less severe with searches with fewer hits; also resetting the game list after each search should help. Of course one does not want to do that, but this is probably the reason that I did not notice this ...)

I will do some more testing and hopefully will have a new version which fixes this out in the next few days.


ez4u wrote:
2. Please uncheck "accept duplicates" in the default installation of the "edit database list" dialog. [...]

3. Please uncheck "Use upper-case labels" in the default installation of the "Edit advanced options" dialog. Alternatively (and maybe better) make the default output lower-case labels when the user chooses "wiki" as the Export mode under Export search results since upper-case are special markup in wiki (and L19) diagrams. Casual users (as opposed to hardcore :rambo: ) may be confused by this if they output diagrams and try to use them online.

4. Last (for now anyway!), could you change the type of widget used to select "Move limit" in the lower right corner Option dialog? It is very difficult to select exactly X moves with a mouse because the slider is too short for the number of moves (250). At times in fuseki searches I want to be specific (for example within exactly 7 moves). As my fingers have become less and less flexible with the years. I am less and less able to hit the number I want. A textbox that I could type the number in would be best.


2. and 4. sound reasonable (and will be easy to do). For 3., I think I will have uppercase labels translated to lowercase in the wiki export (at least if the original position does not have both upper- and lowercase letters).

ez4u wrote:
As always though you have my heartfelt thanks for inventing kombilo in the first place and then for all your work over the years. :clap: :tmbup:


Thanks for your kind words, and please keep making suggestions, if you have ideas for improvement.

Ulrich

_________________
u-go.net

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #5 Posted: Thu Nov 03, 2016 4:04 pm 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
aeb wrote:
ugoertz wrote:
Kombilo is a free go database program. Feedback is appreciated.

I started kombilo on a 100000-game tree, but it did not succeed importing that, it freezes halfway. (Some years ago I debugged this situation and found that kombilo would get stuck after opening all available file descriptors. This time I just observed that the kombilo process had 1024 fds open, maybe still the same problem.)


Thanks for the feedback (and thanks for emailing me about this previously). It is still the same problem (namely that the limit on the number of open folders is reached). For the current version I had decided to "adress" this by putting a note in the documentation (the last paragraph in that section; admittedly this is not easy to find):

Quote:
Kombilo creates a separate database for each folder of SGF files (not including the files in subfolders - each subfolder will get its own database). This causes problems if you have your files in very many folders (hundreds, or more), because at certain points Kombilo will try to open all database files at the same time (e.g., when checking for duplicates). It is better, also for the search performance, to put your games into relatively few folders. It should be no problem to have tens of thousands of games in one folder digested by Kombilo. (As a workaround upon encountering this problem, you could also increase the allowed number of open files; on linux systems, the ulimit command allows to do this.)


However, thinking again about this issue, I guess it would be better to change the Kombilo behavior, and to add all files below some folder, including all subfolders, into just one database. This should not be much work, and would also give better performance if the sgf files are in many small batches in different folders. Of course, it would mean that is not possible anymore to temporarily "disable"/exclude individual folders, but I guess that's not a real issue.

Ulrich

_________________
u-go.net

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #6 Posted: Thu Nov 03, 2016 6:23 pm 
Lives in gote
User avatar

Posts: 301
Location: Illinois
Liked others: 228
Was liked: 84
Rank: infant
ugoertz wrote:
Of course, it would mean that is not possible anymore to temporarily "disable"/exclude individual folders, but I guess that's not a real issue.

Ulrich


I haven't looked through your code, but why couldn't you add a table to your database which has an ID for each SGF and an entry for each folder (and all parent folders) that SGF is located in. Then you can toggle visibility by limiting searches on column B.

It's not super straightforward, but it's absolutely doable in a regular SQL-based DB.

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #7 Posted: Fri Nov 04, 2016 8:41 am 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
Drew wrote:
ugoertz wrote:
Of course, it would mean that is not possible anymore to temporarily "disable"/exclude individual folders, but I guess that's not a real issue.

Ulrich


I haven't looked through your code, but why couldn't you add a table to your database which has an ID for each SGF and an entry for each folder (and all parent folders) that SGF is located in. Then you can toggle visibility by limiting searches on column B.

It's not super straightforward, but it's absolutely doable in a regular SQL-based DB.


That's right, it's doable - but not in ten minutes, in view of how things are currently set up. I do have some plans for restructuring things; hopefully I will get there.

Ulrich

_________________
u-go.net


This post by ugoertz was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #8 Posted: Sat Nov 05, 2016 6:12 am 
Dies with sente
User avatar

Posts: 71
Liked others: 10
Was liked: 19
Rank: 20 kyu
Thanks first for the long time involvement and development of kombilo.
I was using 0.7.4 under wine on Mac OSX

Tried to install 0.8.1 for a native version on Mac OS X:
I use macports: Installed packages:
boost @1.59.0_2+no_single+no_static+python27+universal (active)
python27 @2.7.12_2+universal (active)
sqlite3 @3.15.0_0+universal (active)
py27-tkinter @2.7.12_0 (active)
py27-pip @8.1.2_0 (active) --> set as pip
PATH:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/TeX/texbin:/opt/local/include:/opt/local/include/boost

Install attempt:
pip install kombilo
Collecting kombilo
Using cached kombilo-0.8.1.tar.gz
Requirement already satisfied (use --upgrade to upgrade): configobj in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from kombilo)
Requirement already satisfied (use --upgrade to upgrade): Pillow in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from kombilo)
Requirement already satisfied (use --upgrade to upgrade): Pmw in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from kombilo)
Requirement already satisfied (use --upgrade to upgrade): six in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from configobj->kombilo)
Installing collected packages: kombilo
Running setup.py install for kombilo ... error
Complete output from command /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -u -c "import setuptools, tokenize;__file__='/private/var/folders/nh/v9cbqhcx2_9bqvty0n1lsp180000gr/T/pip-build-B_23h_/kombilo/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/nh/v9cbqhcx2_9bqvty0n1lsp180000gr/T/pip-XRHh6a-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.macosx-10.10-x86_64-2.7
creating build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/__init__.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/abstractboard.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/board.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/custommenus.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/kombilo.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/kombiloNG.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/libkombilo.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/option_editor.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/sgf.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
copying kombilo/v.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/tests
copying kombilo/tests/__init__.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests
copying kombilo/tests/test_final_position_auto.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests
copying kombilo/tests/test_kombilo_startup.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests
copying kombilo/tests/test_sigsearch.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests
copying kombilo/tests/test_v_startup.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests
copying kombilo/tests/util.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/tooltip
copying kombilo/tooltip/__init__.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tooltip
copying kombilo/tooltip/tooltip.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tooltip
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/vsl
copying kombilo/vsl/__init__.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/vsl
copying kombilo/vsl/vl.py -> build/lib.macosx-10.10-x86_64-2.7/kombilo/vsl
copying kombilo/default.cfg -> build/lib.macosx-10.10-x86_64-2.7/kombilo
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/data
copying kombilo/data/references -> build/lib.macosx-10.10-x86_64-2.7/kombilo/data
copying kombilo/license.rst -> build/lib.macosx-10.10-x86_64-2.7/kombilo
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/123.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/abc-l.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/abc-u.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-bookmark-new.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-document-new.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-document-open.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-edit-clear.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-edit-cut.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-edit-find.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-edit-select-all.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-edit-undo.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-go-home.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-go-next.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-go-previous.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-list-add.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-list-remove.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-media-playback-back.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-media-playback-pause.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-media-playback-start.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-media-seek-backward.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-media-seek-forward.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-media-skip-backward.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-media-skip-forward.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-process-stop.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/actions-system-search.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/apps-help-browser.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/apps-office-calendar.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/b.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/black.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/board.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/bw.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/devices-input-mouse.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/go-home-start.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/kombilo_logo.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/places-user-trash.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/sq.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/status-dialog-error.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/tr.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/w.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/wb.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white0.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white1.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white10.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white11.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white12.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white13.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white14.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white15.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white2.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white3.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white4.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white5.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white6.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white7.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white8.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
copying kombilo/icons/white9.png -> build/lib.macosx-10.10-x86_64-2.7/kombilo/icons
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Agon-Agon02-2.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Agon-Agon23-P02.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T01.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T02.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T03.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T04.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T05.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T06.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T07.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T08.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T09.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T10.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T11.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T12.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T13.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T14.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T15.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T16.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T17.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T18.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T19.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T20.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T21.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T22.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T23.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T24.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T25.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T26.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T27.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T28.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T29.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Gosei-Gos23-T30.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/KK1st-KK58-15.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/Oza-Oza54-Oza-2006-r2b.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
copying kombilo/tests/sgfs/under-the-stones.sgf -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/sgfs
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/db
copying kombilo/tests/db/.keep -> build/lib.macosx-10.10-x86_64-2.7/kombilo/tests/db
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/libkombilo
copying kombilo/libkombilo/abstractboard.h -> build/lib.macosx-10.10-x86_64-2.7/kombilo/libkombilo
copying kombilo/libkombilo/algos.h -> build/lib.macosx-10.10-x86_64-2.7/kombilo/libkombilo
copying kombilo/libkombilo/pattern.h -> build/lib.macosx-10.10-x86_64-2.7/kombilo/libkombilo
copying kombilo/libkombilo/pstdint.h -> build/lib.macosx-10.10-x86_64-2.7/kombilo/libkombilo
copying kombilo/libkombilo/search.h -> build/lib.macosx-10.10-x86_64-2.7/kombilo/libkombilo
copying kombilo/libkombilo/sgfparser.h -> build/lib.macosx-10.10-x86_64-2.7/kombilo/libkombilo
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/cs
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/cs/LC_MESSAGES
copying kombilo/lang/cs/LC_MESSAGES/kombilo.mo -> build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/cs/LC_MESSAGES
copying kombilo/lang/cs/LC_MESSAGES/kombilo.po -> build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/cs/LC_MESSAGES
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/de
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/de/LC_MESSAGES
copying kombilo/lang/de/LC_MESSAGES/kombilo.mo -> build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/de/LC_MESSAGES
copying kombilo/lang/de/LC_MESSAGES/kombilo.po -> build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/de/LC_MESSAGES
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/en
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/en/LC_MESSAGES
copying kombilo/lang/en/LC_MESSAGES/kombilo.mo -> build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/en/LC_MESSAGES
copying kombilo/lang/en/LC_MESSAGES/kombilo.po -> build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/en/LC_MESSAGES
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/eo
creating build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/eo/LC_MESSAGES
copying kombilo/lang/eo/LC_MESSAGES/kombilo.mo -> build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/eo/LC_MESSAGES
copying kombilo/lang/eo/LC_MESSAGES/kombilo.po -> build/lib.macosx-10.10-x86_64-2.7/kombilo/lang/eo/LC_MESSAGES
running build_ext
building 'kombilo._libkombilo' extension
creating build/temp.macosx-10.10-x86_64-2.7
creating build/temp.macosx-10.10-x86_64-2.7/kombilo
creating build/temp.macosx-10.10-x86_64-2.7/kombilo/libkombilo
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c kombilo/libkombilo/sgfparser.cpp -o build/temp.macosx-10.10-x86_64-2.7/kombilo/libkombilo/sgfparser.o -O3 -I. -fopenmp
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c kombilo/libkombilo/abstractboard.cpp -o build/temp.macosx-10.10-x86_64-2.7/kombilo/libkombilo/abstractboard.o -O3 -I. -fopenmp
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c kombilo/libkombilo/search.cpp -o build/temp.macosx-10.10-x86_64-2.7/kombilo/libkombilo/search.o -O3 -I. -fopenmp
In file included from kombilo/libkombilo/search.cpp:26:
kombilo/libkombilo/pattern.h:34:10: fatal error: 'boost/unordered_map.hpp' file not found
#include <boost/unordered_map.hpp>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1

----------------------------------------
Command "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -u -c "import setuptools, tokenize;__file__='/private/var/folders/nh/v9cbqhcx2_9bqvty0n1lsp180000gr/T/pip-build-B_23h_/kombilo/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/nh/v9cbqhcx2_9bqvty0n1lsp180000gr/T/pip-XRHh6a-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/nh/v9cbqhcx2_9bqvty0n1lsp180000gr/T/pip-build-B_23h_/kombilo/
You are using pip version 8.1.2, however version 9.0.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Comment:
--> /boost/ is in my PATH
--> pip version 9.0.0 is not available in Macports (anyway I don't think the error comes from there).


Also tried to install under wine (uptodate version) :

wine kombilo-081.exe

Gives the following error message: "This program does not support the version of Windows your computer is running"

Thank you for your help !

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #9 Posted: Sun Nov 06, 2016 9:44 am 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
Thanks for looking into installation on a Mac!

pleiade67 wrote:
Tried to install 0.8.1 for a native version on Mac OS X:
I use macports: Installed packages:
boost @1.59.0_2+no_single+no_static+python27+universal (active)
python27 @2.7.12_2+universal (active)
sqlite3 @3.15.0_0+universal (active)
py27-tkinter @2.7.12_0 (active)
py27-pip @8.1.2_0 (active) --> set as pip


Sounds good. (There seems to be an issue with the preinstalled Tcl/Tk packages on Mac OS, see these notes on the Python web site. If you do not yet have the ActiveTcl packages, you should probably install them. But the current problem lies elsewhere.)

Quote:
Code:
pip install kombilo
[...]

    kombilo/libkombilo/pattern.h:34:10: fatal error: 'boost/unordered_map.hpp' file not found
    #include <boost/unordered_map.hpp>
             ^
    1 error generated.
    error: command '/usr/bin/clang' failed with exit status 1


The problem here is that the boost header files are not fou nd. (The pip version certainly is not an issue.) I will try to have the setup file of 0.8.2 (which wll be out soon) to look in the right place on a Mac, but I could not try it, so it remains a bit speculative.

Quote:
Comment:
--> /boost/ is in my PATH
--> pip version 9.0.0 is not available in Macports (anyway I don't think the error comes from there).


The PATH (for executable programs) is not the path where the compiler looks for these library files.

Quote:
Also tried to install under wine (uptodate version) : [...]
Gives the following error message: "This program does not support the version of Windows your computer is running"


OK, I will set the minimum required version in the installer for 0.8.2 to an older Windows version (I did not think about wine when I first set it.) You could also try to change the version number which wine reports to a higher value (from a quick google search, that seems to be possible using winecfg, but I did not have a go at it myself).

If you are willing to use wine, you could also try to install Python 2.7 in wine, and then use pip to install kombilo with something like
Code:
c:\python27\scripts\pip.exe install kombilo


Since there is a "wheel" available for Kombilo on Windows, there is no need to compile the C++ extensions (so, no need to install sqlite and boost - it is all bundled in the wheel), and with some luck it might work that way. If it succeeds, you could try to start kombilo as
Code:
c:\python27\scripts\kombilo.exe


However, I definitively would hope that we will eventually get it working natively on Mac OS. I will add this to the to-do list with a higher priority ...

Best, Ulrich

_________________
u-go.net

Top
 Profile  
 
Offline
 Post subject: Kombilo 0.8.2
Post #10 Posted: Sun Nov 06, 2016 1:56 pm 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
Hello! I just released a new version, Kombilo 0.8.2. See the link for install/upgrade instructions. Upgrading should be easy and does not require a reprocessing of the SGF files.

The main changes are
  • Better memory management in several places. The memory footprint is now much more stable than in 0.8.1. Even the memory usage on startup should be reduced a little bit (but remains substantial, of course, since some of the databases for the pattern search are read into memory on startup). I expect that this addresses the issues that you, Dave (ez4u), had (although I must also say that I was not able to reproduce such a quick increase in memory usage as you had).
  • Better handling of collections of SGF files coming in many folders. You can now optionally add all files below some folder recursively into one database. Using this options, it is no problem to add files in a directory structure with many folders as in aeb's collection. (Note that this is not the default - you have to choose this option explicitly.)

Let me elaborate a little on how to structure your collection to make Kombilo work most efficiently: I recommend to aim at something between 10000 and 100000 games per database. This also depends on how much RAM your computer has, of course. Processing a database of 100000 games will require around 1.5 gigabytes of RAM available for Kombilo. I was able to build a Kombilo game list of 1.6 million games (with around 100 databases, mostly with 10000 to 20000 games (KGS games), on my laptop with 8GB RAM without any problems). With so many games, the startup time is quite noticeable, and also resetting the game list after doing searches is not immediate. However, all in all I found the response times still quite acceptable. On startup, Kombilo uses a little less than 4 GB of memory, and this remains pretty much stable, even after many searches. Next week I will set this up on a faster machine and build a large "joseki dictionary" using the new SGF tree function.

As always, your feedback is appreciated.

Ulrich

_________________
u-go.net


This post by ugoertz was liked by 3 people: aeb, Bonobo, Drew
Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #11 Posted: Mon Nov 07, 2016 2:25 pm 
Dies with sente
User avatar

Posts: 71
Liked others: 10
Was liked: 19
Rank: 20 kyu
Hello, good news for Mac users:
kombilo 0.8.2 buils under Mac OS X 10.10

The steps in some details (using MacPorts):

Install the following packages:

sudo port install boost —> Collection of portable C++ source libraries
sudo port install python27 —> An interpreted, object-oriented programming language
sudo port install sqlite3 —> an embedded SQL database engine
sudo port install py27-tkinter —> Python bindings to the Tk widget set
sudo port install py27-pip —> A tool for installing and managing Python packages
sudo port select --set pip pip27

Build kombilo with pip:
pip install kombilo

the binary kombilo is located in:
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/

------
Windows version using wine also works:
wine kombilo-082.exe
(the previous version must be uninstalled)

I have added my 2013 GoGoD Database.
Everything apparently went smoothly. Will test the SGF tree tomorrow.

Thanks again for your outstanding software :clap:

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #12 Posted: Tue Nov 08, 2016 1:15 pm 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
pleiade67 wrote:
Hello, good news for Mac users:
kombilo 0.8.2 buils under Mac OS X 10.10

The steps in some details (using MacPorts): [...]


That's great to hear, thanks for trying and writing down everything step by step (and also to Marcel for the alternative using homebrew). I will add this to the documentation.

Please keep me updated if there are any other problems. (Am I correct in assuming that Macs now can "simulate" a right mouse click? There is the "Only one mouse button" option in the Advanced Options as a work-around, and I could activate this by default when installing on Macs.)

One other thing I will have to look into: Currently I "switched off" the parallelization in the Mac OS settings, i.e., only one CPU core will be used for the pattern search. (From googling, I had the impression that this might not work out of the box, and I thought it would be better to get the core working before adding this further stumbling block. Technically, the OpenMP library is required.)

Anyway, now that the program is running in principle, I am pretty sure that all other issues can be sorted out.

Ulrich

_________________
u-go.net

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #13 Posted: Tue Nov 08, 2016 2:06 pm 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
Marcel Grünauer wrote:
ugoertz wrote:
Please keep me updated if there are any other problems. (Am I correct in assuming that Macs now can "simulate" a right mouse click? There is the "Only one mouse button" option in the Advanced Options as a work-around, and I could activate this by default when installing on Macs.)


Right-clicking doesn't work for me. This is on a MacBook Air, so I'm using Ctrl-clicking the trackpad to simulate a right-click; this works in all other apps (Chrome, Terminal, VLC etc.). But Ctrl-clicking deletes a stone, as set in Edit > Ctrl-Click behavior.


OK, I see. If you go to "Edit advanced options" in the Options menu, and set the "Only one mouse button" option as suggested there, can you mark a region of the board with Alt+(click and drag)?

Quote:
This StackOverflow post says something about '<Button-2>' vs '<Button-3>'; could it be related to that?


Kombilo does bind to Button-2 (which is usually right-click), but it also binds Ctrl+Left click, so there is a conflict with Ctrl+Left click as a simulation of right click.

Quote:
Also, the icons are wrong; they're too small and also positioned in the bottom-right corner of their container. Here is a screenshot.


Oops ... thanks for the screen shot. I suppose this is a retina display? Could you check whether increasing or decreasing the "Icon size" option (Edit advanced options; Various options section) improves the situation?

Thanks for the feedback,

Ulrich

_________________
u-go.net

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #14 Posted: Tue Nov 08, 2016 2:38 pm 
Dies with sente
User avatar

Posts: 71
Liked others: 10
Was liked: 19
Rank: 20 kyu
MacBook Pro retina 15-inch mid-2014 OS X 10.10 kombilo 0.8.2 :
- Right click works for selecting the pattern search area (although I prefer connecting a two button mouse)
- I don't observe the icon display problem, they are rendered properly.

(Nothing is simple, everything is complicated ;-) )

Best

Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #15 Posted: Fri Nov 11, 2016 3:24 am 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
Hi Marcel,

thanks again for the detailed feedback. I can now reproduce the icon problem (and found out how to fix it). Also, I noticed that the "only one mouse button" option does not work properly in 0.8.2, so it is no surprise that you saw no effect - sorry! I will change it so that optionally you can select the search region by click and drag with the "standard"/left mouse button, and make that the default on Macs.

I will do some more tests and hopefully release a new version on the weekend. Not sure whether I will be able to do something about enabling the use of several CPU cores in the pattern search. But apart from that everything should work then.

Ulrich

_________________
u-go.net


This post by ugoertz was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Kombilo 0.8.3
Post #16 Posted: Sun Nov 13, 2016 9:40 am 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
The new version is out, and should run much more smoothly on Macs with homebrew (icon appearance and "only one mouse button" option fixed). Selecting the search region on Macs now (by default) works using the standard ("left") mouse button and dragging. You can switch between this and the Kombilo default (right-click+drag) by (de-)selecting the "only one mouse button" in the advanced options editor (Options menu -> Edit advanced options).

It does not look as nice as a native application, but that is hard to achieve with a cross-platform framework ... but the functionality should be there, as far as I can see.

I also added some further small improvements (thanks to people sending suggestions), and references to the commentaries on the Go Game Guru site. So upgrading is recommended. It is not required to reprocess the SGF databases (but if you want to use the "Anywhere" search field in the game info search, you will have to do it, because that was broken in version 0.8.2).

Further information, and download link for the Windows installer: http://u-go.net/kombilo/

As usual, feedback is appreciated.

Ulrich

_________________
u-go.net


This post by ugoertz was liked by: pleiade67
Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.4
Post #17 Posted: Sat Nov 26, 2016 8:57 am 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
I just uploaded version 0.8.4 which has a number of small improvements over 0.8.3, in particular for building sgf trees (e.g., there is now a "Stop button" to interrupt building the tree; and symmetries are handled more correctly).

In principle, computing pattern search trees does work well with large SGF collections. I experimented with around 1.6 million games and more than ten thousand pattern searches in the SGF tree, and will upload one of the resulting "joseki dictionaries" one of these days. In that case, however, it is very much advisable to select the "Reset game list" option. Otherwise on quitting the program, cleaning up the databases will take pretty long.

Download and further information

Ulrich

_________________
u-go.net


This post by ugoertz was liked by 3 people: Bonobo, emeraldemon, pleiade67
Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #18 Posted: Sun Nov 27, 2016 11:02 am 
Dies with sente
User avatar

Posts: 71
Liked others: 10
Was liked: 19
Rank: 20 kyu
Dear Ulrich,

Thank you for updating again kombilo.
If you are interested in distributing a Mac version of kombilo which could be used by everyone, without installing anything else,
I can build a self-contained Mac OSX .app version from the windows exe using winebottler. The .app will contain all the required libraries as well as wine (the only drawback is that the file will be quite large : 300 MO).

Best,
--


Last edited by pleiade67 on Sun Dec 04, 2016 12:20 pm, edited 1 time in total.

This post by pleiade67 was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #19 Posted: Mon Nov 28, 2016 9:38 am 
Dies in gote
User avatar

Posts: 63
Liked others: 0
Was liked: 40
Dear pleiade67,

pleiade67 wrote:
If you are interested in distributing a Mac version of kombilo which could be used by everyone, without installing anything else,
I can build a self-contained Mac OSX .app version from the windows exe using winebottler. The .app will contain all the required libraries as well as wine (the only drawback is that the file will be quite large : 700 MO).


That would certainly be welcome from my point of view, and if you like, I will be glad to host the file on the kombilo web site.

Some further comments. I have briefly been looking into building a "Python wheel" for Mac OS X. This would facilitate the installation of the native version insofar as compiling the C++ extension would be done when creating the wheel, rather than by the user during installation. However, the issue of installing (suitable) Tcl/Tk packages would remain, as far as I understand.

One could go on and create a fully self-contained package similar to the Windows package (using a tool like PyInstaller). This would not require Wine nor winebottler and hence would be much smaller, I suppose, but it would take more time to set things up properly. So for the moment I have postponed this, and the winebottler package could be a good alternative. (If you would like to investigate the wheel/pyinstaller route, I would also be glad to discuss this further.)

Best, Ulrich

_________________
u-go.net


This post by ugoertz was liked by: Bonobo
Top
 Profile  
 
Offline
 Post subject: Re: Kombilo 0.8.1
Post #20 Posted: Thu Dec 01, 2016 12:22 pm 
Dies with sente
User avatar

Posts: 71
Liked others: 10
Was liked: 19
Rank: 20 kyu
Dear Ulrich,

I have sent you a PM with a link to a standalone Mac OS X version of kombilo-084.
Really happy to contribute to the distribution of your invaluable software !

Best regards,

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 22 posts ]  Go to page 1, 2  Next

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group