There's been a lot said about the hazards of the "Hive Mind" lately. Jaron Lanier wrote an excellent essay here. Mike Propst touched on it recently in relation to our attempts over at Netscape.

I just wanted to throw my two cents out there on a particular aspect of it, and explain why the hive can be dangerous without some sort of mechanism for followup.



This just isn't true. It's the top result on Page 2 when my name is searched via google. For the non-geeky folks reading this, I'll do my best to explain why this would be such a "horrible" thing to be said.

To digg users, who are notably young and 94% male, this admission would have been evidence of "gaming the system." Gaming is naturally against the spirit of the democratic site, where the better content should win out with more votes. It strikes many of them as unfair to ask people you know to "digg" material that you submitted. In Weblogs, Inc's case, they believe that its an unfair advantage to have colleagues. Oddly, I'm positive everyone of these kids have begged their friends to digg their false accusations, but hey, what's hypocrisy in the face of teenage angst and a cause.

In reality, what was said was in response to the suggestion that Digg has anti-asking-your-friend-tools and we should develop them too. I replied with something funny like "i can assure you that they don't", or "if they do, they don't work very well", or something to that effect. I say this after doing our own research and evaluation, judging the content posted there, and more importantly by simplistic logic. I simply don't think that they have such tools, can't have such tools, and shouldn't have such tools. If they do have these tools, then it's obvious that they suck. Plus, what would it matter to them. Digg wants more people to come to their site. E-mail your friends. Please. From Netscape's perspective, I think it's a losing battle. I'd rather spend our resources on finding ways to innovate within the space. And yeah, e-mail the hell out of your friends. Spread the word.

Anyway, what I'm getting at is that I was misquoted as saying something that I didn't, and now it's attached to my name on google for ever and ever.

Digg.com takes pride in their we-have-no-editors approach. That's their whole jam. That's their cause that they sell to the young teenage male. Ask someone like Ryan Block, the managing editor at Engadget.com, about how he has to defend himself and his organization what seems like once a month. Without linking to any of that nonsense, the kids over there pretty much look for any reason to attack him. He's constantly taken out of context or reported as doing something that is either completely false or not entirely accurate. The problem is, there's no way to address it. It gets posted, it gets indexed, and there's no one to call and pester for its removal or correction. The only responses are buried deep in user comments and probably never seen nor read.

That wouldn't be such a problem if there weren't a large number of people who trust digg. There are even more people who trust the Google Results that index Digg. Like a growing number of employers.

Imagine I was misquoted as saying something worse, maybe something that could actually get me in trouble. Maybe it's time for a promotion, or theres a new boss to report to, or I seek employment with another company, or whatever other scenario you can think of. In the heat of all the anti-netscape sentiment, some 14 year old brat posts a misquote on a "news" site, and now there's zero method for rebuttal. Awesome... An organized, democratic method for slander controlled by the Hive Mind where liability is skewed in the name of social news and lack of editor accountability.

At Netscape, we've tried to develop mechanisms for users to report stories as inaccurate. An on-duty "anchor" (there's someone online 24 hours, 7 days a week) can research the story and alert users to the dispute. Often, it doesn't need to be removed, it simply needs a notice posted addressing the concern for other users to see. It's simple, it's clean. It's still democratic and user's still have a voice. There's just a basic system of checks to keep things in balance. Seems resonable (and proven) enough to me.

There's a lot of issues surrounding the Hive Mind. Maybe this post can help serve as a plain vanilla example of a basic aspect of it. This particular case is small, it's silly, it doesn't even matter -- but it's a simple demonstration of how the unrestricted hive can't be trusted. It's kind of like the Hitchhikers Guide. It's entertaining but it's inaccuracy has potentially dangerous consequences if you put to much trust into it.

As you may have noticed (or read about over on Jason's blog), we pushed through a slew of changes the last couple of days. I'll briefly touch on them, and kind of outline the work that's immediately in front of us.

New Stuff

• Advertisements - The big blocky crazy distracting ones in the middle of lists? On comments and story pages? Yeah, those are gone. Replaced with clean, crisp, google adsense. Hopefully this will be a nice compromise. Andy is our designer, and is the man, by the way. He's been tackling most of the ad issues.
  
• Frame Navigator - We weren't lying when we said it was something we planned to make optional. All the outrage from the technical crowd quickly made it a priority though. Shout out to Tom for getting it in over the weekend!
• Votes - Lots of little issues with votes, so allow me to get geeky for a second. We have servers. Lots and lots of servers. They're in multiple cities. Users are using different servers, talking to different databases, and pulling from different banks of cache (for now at least, each region has it's own distributed cache shared between servers). Each page load you make, things get cached and stored in memory -- this technique drastically reduces our database calls. Here's where it gets tricky. In some circumstances, we're caching the entire page, in others, only portions of the page. How do you keep the votes outside of what's cached, cached seperately so your not making a ton of unnecessary database calls, and distributed in a way where they won't fluctuate between servers? Now toss in ajax calls every 15 seconds that refresh the vote totals (sit on the page, and you'll see the votes change). I think we have this issue nixed, but we've only rolled it on in one specific place on the page -- the main list of stories. We can use this to monitor what's going on and test. If it appears to be working correctly now, we'll roll it out to the rest of the site. The other places should be accurate to roughly a minute or so. I can assure you though, all votes are being counted. It's just a display problem that we're working on.
• Formula changes - Part of the beta process is to figure out the right speed for the front page. We're not digg, we're a portal. We operate on similar principles, but we have a different user base and different traffic requirements. First, the formula was too slow -- the AOL copies Digg story stayed on the front page for days, which, if you're a fellow fan of irony, is simply rad. On the other hand, it's still "old news" in terms of the life of the story. Then we went to fast, and everything on the front page only had a few votes. Part of this is low traffic, part of it is the speed of the page changes. We're getting close though, lots of little tweaks -- Trey Long, just like Bow Wow, is a man amongst men. Even if he does prefer to be called "Baby Trey."
  
• Lots of little bugs: We had a crazy little bug on the comments page that would remove focus inappropriately as you were typing, we had a js error on vote handling that would show up from time to time, we had some advertisement bleed through that andy nixed, Craig introduced elements of formatting into Comments which I'm sure people will be thankful for, and a bunch of other stuff that I can't immediately think of..
  

Other stuff thats a couple steps behind...

• User suggestions - Most of the "cool" visible stuff will probably be based on user feedback -- we get a ton of it, and we read it all. When you log in, in the top right corner, there should be a "feedback" button. Use it and let us know what your thoughts are :) I'd still like to make the "pop up new window" thing an option, but depends on how loud people are about it.
  
• Better, more scalable multi-city approach - Craig's got some cool stuff prototyped in dev that should help us distribute this thing all over the US. What this means is faster response times and the ability to handle the onslaught of traffic thats heading our way when we switch to the full domain. The difficulty isn't so much in the premise of it, but in coordinating all the different folks that need to be involved.
  
• Better vote display - As noted in the above.. I'm hoping to roll out my crazy ass vote stuff all over the place.
• Other possibilities - If we can get the fires under control, we might be able to get Anchor Chat finished and online. The prototype in dev is, quite simply, friggin' amazing. Mad props to Christoph, Andy, Tom and Trey on it. They've done well.

Vander Walhas an interesting post that's in reponse to another interesting post byMatt McAlister. Both seem to suggest popularity scales may be overrated.Many websites rank items via some sort of popularity. Digg.com uses voting mechanisms, everyone seems tohave some sort of "Tag Cloud" going, some measure clicks or email forwards orpage views. The thought is, the more popular an item on the web, the higherquality of that item.

But what about niche topics? Is coverage on these sites being "normalized", asMcAlister writes? On the face of it, popularity is relative to everything else.If I'm really into pet rocks, a really popular pet rock page isn't going tobe popular in relation to, say, Chinese kids singing like a boyband. If I'm searching out specific information within a somewhat obscuretopic, ranking things by popularity would only seem to make it harder tofind.

Vander Wal writes that popularity gets in the way of information seeking.Maybe, but it would seem that keywords intermixed with popularity would helpmake results more relevant naturally. After all, Google seems to handle mysearch for "petrocks" just fine, and they rank largely on popularity (in fact, thats kindatheir whole gig ;). On the other hand, digg.com might not provide the bestmeans for searching out a topic, but is it supposed to? Seems to me digg is asite that allows you to see the most popular information floating around rightthen and there. Sure, it's collecting meta-data, but, I mean, does digg evenhave categories or tags? It's category is "Technology", and it's tag is"popular." The rest of the meta-data is decent, but suffers from occasional lowquality (as opposed to tags, which would balance themselves out if properlyimplemented).

Still, there's plenty of information that I'm sure is thrown to the waysidebecause it's not showing up on the popularity radar. Is there a way to judgeit's content in a way other than popularity? Sure, subjectively by an editorialteam, but that's simply infeasible on a large scale as it would require expertsin a ton of different catagories. Wikipedia meets dmoz maybe? I donno.

Anyway, I think proper search mechanisms in tandem with popularity measurementis a solid approach. Either without the other doesn't quite cut it in terms ofinformation (re)location.

How fitting. The first comment posted to this brand spanking new blog of mine is "Hey Alex! Bring back bored.cc!" Every couple months I get an email or an IM, or now a blog comment, with that same request. It's actually pretty cool, and I totally enjoy hearing from the bored.cc crowd.

I'd love to bring back bored.cc. Starting from scratch, I could make it such a better site. Seeing things like digg.com come along and basically do the same thing in a cleaner way made me understand some of the usability issues we never got past (while making me a tad jealous in the process.. though, I mean, Kevin Rose is the man, so... it's cool.)

For those who never had the chance to play around with bored.cc, it was basically a generic digg (not just tech) but was based around this whack ass RPG metaphor instead of votes (almost in a joking fashion, but it worked).

Users would submit stories, post comments, post jokes, create polls, participate in web cam rankings, etc. etc. All the while, they'd earn experience points for their participation. Naturally, they'd move up levels every so many points. The higher the level, the more moderation power the user had over the site -- approving/rejecting stories (more like slashdot in this fashion, which is what it was modeled after), etc. The entire concept was built around the idea that I, as the admin, would never ever have to do anything. It should run itself in this weird sort of communal way.

It ran for the majority of my college career untouched (literally, coded and rarely touched again, for years... not good practice, but an interesting experiment), slowly growing in users. Never advertised and only based on word of mouth and search engine traffic, we went down with about 1000 users (at one point we had more, but lots of them were spam/crap..so.. doesn't count ;). Traffic wasn't great, but it would have been nice to throw adsense on it ;)

At the end of it all, I guess I was looking for post-college jobs and knew if someone googled me, they might not get the most flattering returns. That was probably what drove me to shut it down. I should have just removed my name from it and let it run.

Anyway, stay tuned, who knows ;)