L19 network graph

[Dusk Eagle]

Inspired by the High-dan game network thread, I decided to try to graph the interaction between members of the forum (i.e. how often two members post in the same thread). It seemed like a fun weekend project, and I had just written a simplified html parser in Scheme for school so it was a good chance to put that to further use.

So, after writing some Python scripts to download threads from the forum to my computer and extract the names of the authors of each post, I had to figure out a way to graph it all. This took by far the longest time, as I had never done anything like this before so I had to research my options. After narrowing it down to igraph (which walleye used in the thread above) and Gephi, I decided Gephi would be easier to use. Finally, after figuring out how to format my data so it could be imported into Gephi, I was able to make the graphs below.

(Note: Click on an image to get the full size)

http://upload.wikimedia.org/wikipedia/commons/3/34/L19_January_png.png This is the graph for January. Every member with at least one post is represented, but a line is drawn between two members only if one member has posted in the same thread another member has at least three times. A larger line corresponds with more posts in common threads. The lines are also color-coded: yellow represents only slight interaction between two members, cyan represents moderate interaction, and purple represents a large amount of interaction. Nodes are sized and color-coded similarly. There is one caveat with this data: the information was gathered from only the first 20 posts of each thread.

http://upload.wikimedia.org/wikipedia/commons/c/ca/L19_alltime_png.png This is the graph for all time. Once again, every member with at least one post is represented, but a line is drawn between two members only if they have 30 posts in common threads.

This information may not be too practically useful, but I found it quite interesting to look at.

[Stable]

It looked to me as if we form basically one population, rather than a split one. That's nice.

[daniel_the_smith]

That's pretty cool!

[freegame]

It looked to me as if we form basically one population, rather than a split one. That's nice.

yup seems like it.
looking at the all time graph it's also clear a small amount (core) of people is responsible for the vast majority of posts.
Of course I already knew this from looking at the member list, but this make shows it in a nice visual way.

I wanted to create some graphs myself this weekend.
But decided to go to participate in a (real life) Go tournament instead.
I ended in second place so I guess I made a good choice

[topazg]

I need a life!

[daal]

I need a life!

Be happy, you have 2.

[Joaz Banbeck]

@Dusk Eagle: Thanks for doing this. Would it be possible to have a variation with the Malkovich games not counted?

[CSamurai]

@Dusk Eagle: Thanks for doing this. Would it be possible to have a variation with the Malkovich games not counted?

I'm sure it would be, but then I'd have almost no connections at all!

[Dusk Eagle]

@Joaz, I should be able to find a bit of time to do that before the end of today.

[Dusk Eagle]

January's graph without Malkovich games.

The alltime graph without Malkovich games. I'm going to speculate that the weakening of the relationship (no pun intended) between Kirby and Magicwand (due to removing all the Malkovich games between them) boosted others relationships proportionally from yellow to cyan.

Enjoy!

[walleye]

A couple of technical questions.

http://upload.wikimedia.org/wikipedia/commons/3/34/L19_January_png.png This is the graph for January. Every member with at least one post is represented, but a line is drawn between two members only if one member has posted in the same thread another member has at least three times.

Do you mean to say you draw a line if they both have one or more posts (in the first 20) in at least three threads? Or, there is at least one thread where they both have three posts (in the first 20)? Or, is it three posts for just one member? In which case, it doesn't matter which one since your graph is undirected, does it?

There is one caveat with this data: the information was gathered from only the first 20 posts of each thread.

Fortunately, most non-Malkovich threads have fewer than twenty posts. Still, it's an annoying limitation.

http://upload.wikimedia.org/wikipedia/commons/c/ca/L19_alltime_png.png This is the graph for all time. Once again, every member with at least one post is represented, but a line is drawn between two members only if they have 30 posts in common threads.

According to what you are doing, a thread is common to two members if they both have one or more posts (in the first 20) in the thread. Right? So what you do is you find all common threads and then count the total number of posts that they have in those threads (only using the first 20 posts in each thread). Then you draw a line if they both have 30 or more total posts thus counted. Is this correct?

[Dusk Eagle]

Do you mean to say you draw a line if they both have one or more posts (in the first 20) in at least three threads? Or, there is at least one thread where they both have three posts (in the first 20)? Or, is it three posts for just one member? In which case, it doesn't matter which one since your graph is undirected, does it?

For each post in a thread, that author gets one "interaction point" with the author of each subsequent post in that thread. A member can not score interaction points with themself.

So for the January graphs, two members have a line between them if they have at least 3 "interaction points" between them. For the all-time graphs, two members must have at least 30 "interaction points" between them to be have a line displayed.

I gathered the forum data by first observing that all the URLs of threads on this forum are of the form:

Code: Select all

http://lifein19x19.com/forum/viewtopic.php?t=%d%

where %d% is an integer. So, I just used a for loop to download each page. But since threads are limited to 20 posts/page for non-logged-in users, and I didn't know how to have the tool I used to download pages use my log-in (where I could change the posts/page preferences), the 20 posts per thread was the best I could do. However, as I write this, I've just thought of a way to do that which I'm almost certain will work, so were I to do this again at some point I should be able to include all posts, not just the first 20.

[schultz]

Hey, late to the party, but wanted to say this is really cool.

And glad to see that I actually made an appearance, even if only for the all-time graphs.