Spotify go insight

[John Fairbairn]

The context was people complaining that Spotify's shuffle randomiser did not seem random, but I thought the final quote was illuminating for go theory, too:

"Our brain is an excellent pattern-matching device," said Babar Zafar, a lead developer at Spotify, in an interview for Tech Tent on the BBC World Service.

"It will find patterns where there aren't any."

[Vesa]

This seems to fit here: https://www.youtube.com/watch?v=BNOQUPmgbnY from Pi the Movie.

Cheers,
Vesa

[tentano]

Just watch the static of an old CRT TV trying to interpret static noise into picture. Eventually faces, animals, plants, toys, whatever your mind could recognize will show up briefly.

Or those Rorschach bathroom tiles, you can see just about anything in those.

Or in experimental data that you've gathered for years which doesn't seem to validate your hypothesis at all.

(maybe that last one is fraud)

[DrStraw]

"It will find patterns where there aren't any."

Sounds like the quality of my go.

[Mike Novack]

"Our brain is an excellent pattern-matching device," said Babar Zafar, a lead developer at Spotify, in an interview for Tech Tent on the BBC World Service.

"It will find patterns where there aren't any."

And that is presumably correct behavior. Only seems silly if we forget the basics. Any pattern recognizer has two important measurements. The "positive percentage", what percentage of the time that there is a pattern to be detected that it is detected. And the "false positive rate", the percentage of time that there isn't any pattern but what is reported.

OK, then we need to consider the "costs" of the two types of error, the "negative percentage" (pattern was there but not reported) and "false positive" (no pattern there but pattern reported. Since what was discussed was animal brains (ours) we make this an animal problem, yes? The pattern (which might or might not be there) is "tiger behind that bush". Too low a positive rate and you get eaten. Too high a false positive rate and you starve (constant interruptions in grazing). Sorry, but I believe that natural selection will have gotten those rates adjusted to the optimal. Presumably our brains have been adjusted the same way over the last couple million years to be optimal for humans.

[tentano]

Yeah, I'm okay with having lots of false positives instead of missing the stalking tiger 1% of the time.

[Splatted]

Unlike Tentano tigers currently aren't a big issue for me.

Edit: I was wrong. So very very wrong.

[ez4u]

"Our brain is an excellent pattern-matching device," said Babar Zafar, a lead developer at Spotify, in an interview for Tech Tent on the BBC World Service.

"It will find patterns where there aren't any."

And that is presumably correct behavior. Only seems silly if we forget the basics. Any pattern recognizer has two important measurements. The "positive percentage", what percentage of the time that there is a pattern to be detected that it is detected. And the "false positive rate", the percentage of time that there isn't any pattern but what is reported.

OK, then we need to consider the "costs" of the two types of error, the "negative percentage" (pattern was there but not reported) and "false positive" (no pattern there but pattern reported. Since what was discussed was animal brains (ours) we make this an animal problem, yes? The pattern (which might or might not be there) is "tiger behind that bush". Too low a positive rate and you get eaten. Too high a false positive rate and you starve (constant interruptions in grazing). Sorry, but I believe that natural selection will have gotten those rates adjusted to the optimal. Presumably our brains have been adjusted the same way over the last couple million years to be optimal for humans.

While our brains may have been optimized to the world where there were a sufficient number of tigers to affect our large scale survival rates, we have meanwhile 'optimized' the world in the opposite direction by hunting tigers to extinction. Therefore it is reasonable to doubt that our pattern recognition is actually appropriate for our current environment, isn't it? Most particularly the dangerous carnivores that threaten us on the Go board are not so kind as to dress up in striped tiger suits!

[EdLee]

apophenia

[Mike Novack]

I of course used the tiger example to discuss how our ratio "correct pattern recognition" vs "false positives" evolved. The lack of sufficient tigers to represent to threat should not be taken as evidence that some false positives better than missing a pattern isn't still important or even that the ratio is far wrong for modern conditions. It is, after all, a general pattern recognizer, not tiger specific.

The way I most consciously notice "false positives" is when driving. How many of you have also experienced first thinking you see a large animal that might come out into the road. Then the image blurs and is resolved to match a different pattern (bush of weird shape, but something stationary that will not dash out into the road ahead of you). Something most common when visibility conditions are not ideal.

OK, maybe you don't have to worry about hitting a bunny rabbit, but is that a roadside rock or a snapping turtle, a bush of weird configuration or a moose.

[tentano]

Heavy snow is the worst when driving.

At least fog has a static background, and any movement not stationary to the ground is clear.

Snow, though? Anything COULD be about to dash out in front of your car. It's a lot like TV static. I don't drive when there is heavy snow because of this. I don't enjoy driving while terrified.