Posts with tag development

Twitter Developer Calls out Ruby on Rails...

I can't tell you how many people have tried to shove ruby on rails down our throats. I'll agree that its pretty looking code, and that its probably pretty manageable, but as of now it is an abosolute nightmare when it comes to scaling.

At SXSW, we had some dude freaking out about how we didn't know what we were talking about because his site could handle hundreds of thousands of users at a time.. It was cute, and almost not even worth responding too given our background with sites like Engadget, TMZ, Autoblog, Joystiq and the rest of the happy blogsmith users.

Anyhoo, one of the Twitter developers, Alex Payne, was interviewed and specifically asked about their experiences with the Rails framework, being that they are now probably the biggest Rails site on the net.
The common wisdom in the Rails community at this time is that scaling Rails is a matter of cost: just throw more CPUs at it. The problem is that more instances of Rails (running as part of a Mongrel cluster, in our case) means more requests to your database. At this point in time there's no facility in Rails to talk to more than one database at a time. The solutions to this are caching the hell out of everything and setting up multiple read-only slave databases, neither of which are quick fixes to implement. So it's not just cost, it's time, and time is that much more precious when people can['t] reach your site.

Full interview here
I have yet to meet a Rails developer who honestly thinks that Rails (in its current form) is a highly scalable framework, unless they come from a design background or a java background. Two backgrounds who ultimately shouldn't be allowed to discuss efficient, affordable scaling in any serious tone. ;)

(hat tip Gavin for the link)

Netscape.com Updates...

As you may have noticed (or read about over on Jason's blog), we pushed through a slew of changes the last couple of days. I'll briefly touch on them, and kind of outline the work that's immediately in front of us.

New Stuff
  • Advertisements - The big blocky crazy distracting ones in the middle of lists? On comments and story pages? Yeah, those are gone. Replaced with clean, crisp, google adsense. Hopefully this will be a nice compromise. Andy is our designer, and is the man, by the way. He's been tackling most of the ad issues.
  • Frame Navigator - We weren't lying when we said it was something we planned to make optional. All the outrage from the technical crowd quickly made it a priority though. Shout out to Tom for getting it in over the weekend!
  • Votes - Lots of little issues with votes, so allow me to get geeky for a second. We have servers. Lots and lots of servers. They're in multiple cities. Users are using different servers, talking to different databases, and pulling from different banks of cache (for now at least, each region has it's own distributed cache shared between servers). Each page load you make, things get cached and stored in memory -- this technique drastically reduces our database calls. Here's where it gets tricky. In some circumstances, we're caching the entire page, in others, only portions of the page. How do you keep the votes outside of what's cached, cached seperately so your not making a ton of unnecessary database calls, and distributed in a way where they won't fluctuate between servers? Now toss in ajax calls every 15 seconds that refresh the vote totals (sit on the page, and you'll see the votes change). I think we have this issue nixed, but we've only rolled it on in one specific place on the page -- the main list of stories. We can use this to monitor what's going on and test. If it appears to be working correctly now, we'll roll it out to the rest of the site. The other places should be accurate to roughly a minute or so. I can assure you though, all votes are being counted. It's just a display problem that we're working on.
  • Formula changes - Part of the beta process is to figure out the right speed for the front page. We're not digg, we're a portal. We operate on similar principles, but we have a different user base and different traffic requirements. First, the formula was too slow -- the AOL copies Digg story stayed on the front page for days, which, if you're a fellow fan of irony, is simply rad. On the other hand, it's still "old news" in terms of the life of the story. Then we went to fast, and everything on the front page only had a few votes. Part of this is low traffic, part of it is the speed of the page changes. We're getting close though, lots of little tweaks -- Trey Long, just like Bow Wow, is a man amongst men. Even if he does prefer to be called "Baby Trey."
  • Lots of little bugs: We had a crazy little bug on the comments page that would remove focus inappropriately as you were typing, we had a js error on vote handling that would show up from time to time, we had some advertisement bleed through that andy nixed, Craig introduced elements of formatting into Comments which I'm sure people will be thankful for, and a bunch of other stuff that I can't immediately think of..
Other stuff thats a couple steps behind...
  • User suggestions - Most of the "cool" visible stuff will probably be based on user feedback -- we get a ton of it, and we read it all. When you log in, in the top right corner, there should be a "feedback" button. Use it and let us know what your thoughts are :) I'd still like to make the "pop up new window" thing an option, but depends on how loud people are about it.
  • Better, more scalable multi-city approach - Craig's got some cool stuff prototyped in dev that should help us distribute this thing all over the US. What this means is faster response times and the ability to handle the onslaught of traffic thats heading our way when we switch to the full domain. The difficulty isn't so much in the premise of it, but in coordinating all the different folks that need to be involved.
  • Better vote display - As noted in the above.. I'm hoping to roll out my crazy ass vote stuff all over the place.
  • Other possibilities - If we can get the fires under control, we might be able to get Anchor Chat finished and online. The prototype in dev is, quite simply, friggin' amazing. Mad props to Christoph, Andy, Tom and Trey on it. They've done well.

Extending SMS...

There's an interesting post on O'Reilly Radar regarding Mozes,a new SMS service.

Gavin and I have been playing alot with our cell phones. I'm a big java guy, so the idea ofdropping on an application to interface with some of our web applications is appealing. Once youget into all of it though, you uncover a world of mess. The cellphone space is a complete cluster-f of horribly supported,unstandardized APIs geared at allowing people to write yet anotherPacMan clone. On top of it, Verizon doesn't even use Java. They useBREW, which is basically an awful way of keeping people from easilydeveloping applications for their halfassed phones.

What I'm getting at is, SMS is really the only viable option for acell phone product right now. The location aware technologies thatare needed to do anything really neat are just too far away. Notjust that, they seem to cause so many ridiculous privacy concernsit's almost not even worth dreaming about. Hopefully the cell phonemanufacturers and the cell services in this country will get theiracts together at some point though, because thats "the next bigthing."

Anyway...

Mozes is a great idea. It'spainfully simple, and frustrates us to know how close we came towriting something similar in our prototyping. We built a reallycool SMS framework a few weekends ago to see if SMS could be aviable alternative to some of the things we're looking to build.The short of it is, yes, yes it is.

There's something to be said for developing for the least commondenominator ;)