Personally, I don't really buy the argument about using MD5 without any worries at all. There have been enough exploitations of MD5 to create a concern when you're using it for an important application. A popular example is this one: http://www.win.tue.nl/hashclash/rogue-ca/. However, it is true that these people were actively trying to break the system. It's also against my philosophy to develop something with a known problem if it can be avoided (like in this case by not generating filenames at all)...

But you guys are right: there is a problem with the current implementation if the characters in the URL get too long. Also as has been said, if a collision did occur, we could address it at that time. And the probability of a collision is still low, especially if nobody is trying to attack the system. Another point is that it seems that diagrams have already been implemented in this manner without problems, so it might be OK to follow suit...

It's not as straightforward as copying what Adrian has done verbatim because of the limitations in the BBCode, but we should probably go this route. I think that the possibility of MD5 collisions is low enough that it is a much lesser issue to worry about than passing things through the URL like this.

So if we do go that route, what about using php's sha1 function? It's unlikely that we'll have an MD5 collision, and probably even more unlikely that we'll have a sha1 collision. It probably doesn't make a difference, since we'll probably get no collisions at all, but it might make me feel a little happier inside.

SHA1 is a little longer than MD5, but also a little slower. Neither of them will generate a collision with any likelihood at all.

To put this in perspective: If you post 1 diagram every second for 1 billion years, continuously, then at the end of that billion years you will have roughly a 1 in a million chance that two of those diagrams have the same MD5 hash (you will have posted over 30 million billion diagrams at that point). I think we have more important things to worry about

I already know that the chances are very low - I just don't feel good about it philisophically. Even if the URL problem were the only issue with the current implementation, I think it's enogh of a reason to go with the file cache, though. Mainly, the current implementation is just what we got to work with bbcode first.

I wasn't aware of the amount of time it takes to generate a particular image for a single request. Do you have any idea on the magnitude we're talking about?

I have no idea. You could benchmark it if you want, but the exact numbers really aren't really relevant. The point is that you're doing the same thing hundreds of times when you really only need to do it once.

That's true. I guess it in that sense, it's a tradeoff between extra work in creating an image, and extra space being taken up on the server by saving the image. Considering the points that have been brought up in this thread, though, I'm still in agreement that we should probably save the images to the server.

A quick note on this subject: the way GoDiscussions operated was to take an md5 hash of the SGF or Go Diagram text, see if a file existed on the server with that md5 hash, create the file if not, and then use that url in the generated html. The first server "crash" where I started getting involved (I think a couple of years ago now) was because there was a single directory with bazillions of these md5 files, and the server's filesystem couldn't handle it. If you go this route, I'd recommend partitioning the files somehow (e.g. by the first character or two of the md5 hash) to make it easier on the filesystem. (That's how I fixed GoDiscussions that time around.)

Thanks for the tip, Ross. All things taken into consideration, is this the route you would take if you did it yourself? Since you said "If you go this route", I'm wondering if you have some other ideas that we haven't considered, yet.

I had actually never considered generating the image on every page load (like you do now) when Adrian and I were working on hacking the phpbb3 code to save the files on the server GD-style. It's very creative, and after seeing it work fairly well, I'm not convinced that the extra server load is significant. However, I think the annoyances of e.g. filenames when you download and other small things make it worth it to switch to the md5sum route.

Oh, another small warning—GoDiscussions had some code that attempted to "phase out" older diagrams (i.e. enabling them to be removed from the server once they were X many months old). The code never worked and actually ended up making multiple unnecessary copies of the entire md5 collection of images and sgfs (another reason why the server tanked), but I just wanted to caution you that this is probably not only unnecessary but possibly counterproductive—old threads are going to be looked at all the time, both by humans and (e.g search engine) bots, so trying to save space by deleting old images isn't going to work very well. You'll probably have to keep them all around until the end of time (which is why the partitioning approach you take is so important). Just another friendly hint.

...

Caching the images server-side is a good idea for performance reasons even if you continue to use the javascript method.