Future of Web Apps Conference 24 comments
Last week I made a spontaneous trip to the Future of Web Apps conference in London. I wasn’t planning to go, but I’m glad I did…there were many great talks and I learned a lot about the greater web development community, especially people’s impressions about Ruby on Rails.
It had the highest ratio of quality talks of any conference I’ve been to. Here are my notes from a few of the most useful talks.
Steve Souders, Yahoo, High Performance Web Sites
Yahoo’s performance expert (and recent author) talked about performance. Not a word was said about MySQL optimization or profiling your server-side code…it was all about increasing the subjective page load on the client.
As server-side web developers, we would like to think that performance is all about making SQL queries more efficient and making our Ruby code faster. The theory presented was that a user’s impression of the speed of a site depends much more on things like compressed content, placement of script include tags, and the geographic location of the content being served. He also mentioned using a content delivery network, but this is quite expensive for a small site (I’ve heard that an introductory account with Akamai starts at US$10k per month).
Over the past year or two, I’ve used some of these techniques on this blog and other sites. Rails-specific resources:
- Nginx config file with gzip compression turned on
- Scott Becker’s Asset Packager (a subset of this functionality will be available in Rails 2.0).
- Ezra Zygmuntowicz’s balanced assets (something similar will be in Rails 2.0).
The YSlow plugin for FireBug can also help analyze a site for the issues mentioned.
Matt Mullenweg, Wordpress
The author of Wordpress talked about scaling their blog-hosting service to the point of being one of the top 25 most visited sites on the net.
Notable points:
- They use software load balancers exclusively. Pound and Whackamole (I couldn’t find a url), Varnish for caching assets. The also use the Spread messaging server, which I need to learn more about.
- He mentioned a video from a Wordpress conference that is well worth a watch. Summary: On a given dedicated server, a default Wordpress installation only does 8 requests per second. VM-based caching can bring that to 12, but other caching plugins pull it up to 300 requests per second. Lesson: Fragment caching and page caching can provide a much greater benefit than even database optimization.
- The HyperDB database adapter. This thing is a drop-in replacement for the PHP database adapter and does things like sharding content and tables among database servers and connecting to backup databases when the master is unavailable. Most sites don’t need anything this robust, but it seems that it would be very possible to implement for ActiveRecord. The code is only about 600 lines long and is open source.
- The Wordpress.com database is split into 4096 databases on 8 physical servers. They take an MD5 hash of the blog name and map the first 3 characters to a database. Although he admitted that 4096 was a bit of overkill, they will be able to spread the databases to more physical hardware with only a minor configuration change in their application.
- HyperDB also does a fair amount of benchmark logging as well. Rails equivalents for logging: query_trace, query_analyzer. I committed some code to the current Merb trunk so it can report the time run for each
beforefilter. It would be nice to have something similar for Rails, too. - Their deployment system is just a script that runs
svn updateon all 300 of their servers. At one point they had to replicate their repository because it couldn’t stand up to that many hits at once.
Interestingly, Matt answered a question about client-side optimization and felt that a content delivery network advocated by Steve Souders was not worth the work. They serve Wordpress.com assets from three servers in Texas, and also from Amazon S3.
Matt Biddulph, Dopplr
Matt Biddulph gave a useful talk on implementing Dopplr. A few interesting points:
- Amazon S3 is fairly cheap for storage, but expensive for bandwidth. They use this rake task to backup the database to S3. I’ve been using the same script to backup PeepCode and to offer an alternate download for PeepCode files. Last month I stored 9 GB and transferred 50 GB and paid US$10, so it was quite affordable.
- Dopplr has been experimenting with using S3 and EC2 to mirror their database as an instant backup solution. He advocated configuring a slave database with MySQL replication even if your web application never touches that database.
- They wrote and open sourced identity matcher for slurping in lists of friends from other social networking sites. Unfortunately, this requires users to give you their name and password for third party sites, but he’s hoping to solve that with oAuth and OpenID.
- He suggested running both an OpenID client and server application on your site in order to let users access their data (I recently implemented OpenID signup and login at PeepCode). I’m confused about how this would work, but I guess that it would be something like:
- Login to Dopplr with OpenID
- Enter my OpenID url for another site, i.e. topfunky.somesocialnetwork.com
- Use sreg from Dopplr to send you to SomeSocialNetwork, which sends back a list of your friends or other data that you have approved of.
- Unobtrusive Javascript Widgets. Instead of doing a
document.write, Dopplr uses the shortloaded Javascript snippet to load Javascript-generated content into adivon the client’s site. So you can use the ideas mentioned above and speed up client-side load time by including the Dopplr widget anywhere on your page.
On another note, I was surprised to see S3 download speeds in excess of 250kb/second from London. I thought they would limit the speed at which files are served, but apparently that’s not the case. Even so, I’ve bought a small VPS in Australia and am nearly ready to launch a Merb-based asset server for serving PeepCode downloads to Australia and Asia.
You Are P.R.
On the Rails front, it was shocking to hear that Yahoo is using Rails for their code-named “FireEagle” geotracking service after their commitment to use PHP for everything. Maybe it’s just for the prototype? It’s sure to get a ton of traffic when it launches.
The most frequent Rails-related comment I heard from people at the conference was about Derek Silvers’ post on Why I Switched back to PHP. Most people probably read the title, concluded that Rails wasn’t worth learning, and went about their business (at least that’s the content of most of the comments I heard at the conference).
I have a ton of respect for Derek and I think the article hasn’t been properly understood by most people. However, as a developer for an open source product, you are the only public relations department available. In an upcoming interview for the Rails podcast, James Cox talks about how PHP had to intentionally think about the public image of the language. They actually took steps to make sure that accurate information was being communicated instead of only the headline-worthy news.
Where will this come from for Rails? The author of Rails is unlikely to become a calm, diplomatic advocate in a way that non-Ruby web developers can appreciate. Heck, even the Seattle.rb has a reputation for promoting their projects in an offensive way. For the core team, it may not matter whether or not Rails is widely adopted, but for those of us who make a daily living using Rails, the public reputation of Rails IS important.
At one point there was something called MINASWAN, but I don’t think that is very well known inside the Rails community (not to mention outside of it).
So is there hope for the Rails PR machine? Is it possible for us to reverse the popular opinion of it as an unscalable, offensively-promoted niche framework?
Nuby on Rails
Geoffrey Grosenbach / Ruby / Code / Graphics / Design / Rails / Merb / Javascript / CSS
Manufactured with
Subscribe
Menu
Recent Articles
- Not with a bang...
- Improve Your Technical Slides
- The Third Seattle.RBBQ
- PeepCode is [No Longer] Looking to Hire a Virtual Assistant
- IOTC Open Source Conference in Dublin
- → Older articles
Speaking At
Elsewhere
- PeepCode Screencasts
- Ruby on Rails Podcast
- Rails Workshop Calendar
- Job Leads
- Task Scoresheet
- Podcasting Basics
- Topfunky on Twitter
- Topfunky Corporation
Free Software
I don’t think there is hope, at this point at least. That is, unless we start really looking at other communities and take the good parts from them instead of pissing on everyone else “because we’re awesome” (that’s the external appearance we give at least).
Of course, we aren’t like most OSS communities in the first place. The whole Rails ecosystem is fairly backwards compared to most open-source projects of its size. There’s very little delegation, no “image” management as you said, very little roadmapping outside of DHH’s brain, and so on. Not that that’s bad, but I don’t think we can really expect to operate like other communities if fundamentally we don’t work like them in the first place.
Even further, some people are douchebags. That’s just their personality (I’m not referring to the people you mentioned necessarily. Go in #rubyonrails sometime…). And sometimes these people get popular or are at the top inherently. That’s not to say that they’re bad people or aren’t nice people sometimes, but their brash image is going to be what people remember (hence why I’ve seen DHH’s infamous slide posted more than any other picture of him). And honestly, I don’t know if there’s a cure for that. In my experience with other projects, those people don’t gain the noteriety they seem to hold in our community, simply because a lot of people don’t like them, so they eventually leave because they aren’t getting the attention they seek. For some reason in the Rails (and to an extent, Ruby) communities, completely opposite from MINASWAN, this sort of attitude is cherished.
I dunno. Maybe we’re stuck until someone rises up and says this stupidity has to stop.
I guess I’d just like to be able to meet other web developers and hear “Oh, Rails? That’s the one with a really great community and a team that is working hard to make it deploy and perform better, right?”
Instead, people write it off immediately or assume that they need to immediately put up their defenses. There’s no discussion.
Heck, even Mac users once had an elitist reputation that’s not much different from the Rails reputation right now, but it seems like that has changed for the better in the past few years.
About your last point – Rails PR – I think the best PR machine would be “the power of silence”. If we all ( rails community ) just STFU in terms of blog posts/comments in response to FUD! Seriously, think about it.
When I read your comment on why DHH is unlikely to become a calm, diplomatic advocate I didn’t agree with you. I felt that being a little head strong isn’t going to hurt anyone.
But then I read his recently blogged Potty Mouths rant, and now I think I’m starting to agree with you.
I don’t think we need to worry.
I’m a long-time Perl developer. I’d been doing web development since 1995. I watched as each new fad came in and out of fashion. Cold Fusion. Java. PHP. DotNet. Python (not that Python is bad).
I passed on them all, because Perl was such a superior tool.
But I couldn’t pass on RoR. It really is that good. If RoR itself doesn’t ultimately win, every other dev environment is going to copy its major features (witness Catalyst in the Perl community—a flawed, if valiant attempt).
There’s enough inertia behind Rails that I don’t think anything can derail it. The community isn’t particularly bad either, BTW (Perl certainly has its problems). I’ve never heard DHH speak, but I think Rails is larger than any individual now.
Certainly there are problems. It is impressively slow. But that will be fixed. It’s too good not to fix.
lol, just went to read the potty mouth post. Googled dhh, clicked the first link, then clicked the gallery to see David’s flickr page. My thoughts, Geek is the new black, David’s a very metro looking fanboy, the internet still fascinates me, I came here to steal your sass hack cause I just generated my first merb app and feel really good about it. I hope it snows soon because my new jib board is giving me blue balls, maybe I should put it in the closet. hmm…
Thanks Geoff for a useful and insightful post. I’m with you on the need of the Rails community to grow up a bit. DHH’s potty mouth post really does show the lack of mature leadership at the top. Rails PR really does affect those of us who use Rails professionally.
About rails PR: I agree strongly with Pratik.
The people who complain about others should lead by example instead. Be nice, positive, mostly quiet, etc.
Where is your nicety and kindness and quietness, people? Where is your respect for others? Especially for those that gave you something?
What this post and comments have shown is that the only difference between DHH and most of his critics is that he is honest, while critics are using cowardly politically-correct weasel language for attacks. There is a word for such people. Hypocrites.
P.S. I am not nice nor quiet, but I do not complain about others being the same. I quite like the DHH style – it is a refreshing and direct approach, same as in Rails.
I started writing a comment, but it got a bit too long
Hi!
I just want to tell you that your post was much appreciated, Geoff. I’m glad you went and brought back those goodies.
Wackamole URL: http://www.backhand.org/wackamole/
Geoff, do you have any concrete ideas? It seems to me that the nature of your business gives you a far more vested interest in the PR image of Rails than the members of the core team, and I’m certain that the community would welcome your help.
The truth is, the core team is far more interested in making Rails work well technically than they are about PR. If you consider the fact that the core team members make their living using Rails to write production applications, not by offering Rails training, it shouldn’t come as a surprise.
At the same time, I don’t think it’s fair to characterize the core team as being indifferent to the issues of community involvement, scaling, and deployability. The entire patch process has been streamlined, and patches that meet the requirements for inclusion are committed nearly every day, performance patches included. As far as deployability is concerned, Jamis works tirelessly to respond to nearly every question that’s posted to the Capistrano mailing list and has gone out of his way to accomodate requests for enhancements.
It seems like the real problem is more one of perception rather than reality. I think part of the solution is to stop feeding into and perpetuating the meme.
PDI ways to improve the perception.
maybe it’s time for a branch…
Alex G – I believe it’s called “Merb”
@Mark Imbriaco said
That’s exactly my point. I think the Rails community and the greater Ruby community have done a lot to improve Rails. But somehow, that message isn’t getting out to the greater web development community.
Initial ideas:
A few years ago, Bill Gates hired a PR firm to improve his public image. He listed things he wanted to portray, like “Bill Gates cares about the community,” “Bill Gates is generous,” etc. I would start with that same strategy. What do we, the community, want Rails to be known for?
And there’s always the valuable one-on-one conversation. In London I talked with someone who felt that the Rails Javascript helpers were underpowered for what he wanted to do. I said “It’s OK…you don’t have to use them. You can code straight against Prototype if you want to. In fact, I do that exclusively.”
Or I even heard “Rails deployment is hard because it doesn’t setup cron jobs for me.” I had never heard this argument, but apparently some Java frameworks have reimplemented cron and handle that for you. The answer was just, “No, Rails doesn’t even try to do that, but you could automate it with the Capistrano deployment tool.”
So there’s no way we’ll be able to address every objection, but Rails can be more accurately represented when intelligent, non-defensive conversations can happen.
We’re using RoR every day at vodpod.com. I’ve used everything else except Python, and Rails beats them all hands down.
Look, if you poll smart developers, Rails has no image problem. What are they working on in Java land? Grails, JRuby, and outfitting Eclipse and Netbeans to support Rails!
What are they doing in PHP? They’re copying Rails in building the Zend Framework.
All the smart developers are either using Rails already, or trying to recreate it in their favorite language! So where’s the image problem? Just because a few sub-pars write some blog posts complaining about DHH’s language does not mean Rails has an image problem. Every one of those clowns is working hard to protect some “expertise” they’ve established, and frankly they don’t want to have to learn something new.
So please, let’s not learn anything from the PHP community! What the Rails community has to teach you is that your time is beter spent building great stuff than trying to manage your P.R.
@topfunky: First part of your post (about explaining stuff to uninformed) is great. It makes a positive contribution. The rest of your post is really uncalled for. Your post illustrates exactly what is wrong and what can be improved – unclear posts which are then used for FUD.
@Scott Persinger: Totally and completely true. Great post!
I think DHH’s keynote at RailsConf ‘07 Europe addressed your PR issue, it was good for business and visibility to be rockstar assholes in the early days of Rails adoption.
You have two camps, the enterprise mind, and the startup mind. Those of who have startup minds appreciate the loud mouth style that has become synonomous with DHH (and therefore rails), and those with the enterprise mindset won’t care anyway until they are drug kicking and screaming into the technology.
Personally I’m not sure that it is damaging to know you are the best framework on the block and stand on top of the Eiffel tower and yell it aloud. As you point out, people said the same thing about Macincultists™ and it didn’t do them any harm.
Their marketshare would be even smaller (due to the business practices of MS) if they hadn’t picked up the hearts of people who like to be different.
And that post about switching back to PHP was filled with BS and bad knowledge, if that is what drives a developer perhaps they should switch to business instead of technology.
Rails is grrrrrrrrrrreeeeeeeeeeaaaaaaaaaaaaaaattttttttttt!
@topfunky – OT but I’m curious, are you still maintaining Gruff? There are useful patches that have been submitted into the rubyforge bug tracker, it would be nice if you could go through and apply those and release a new gem including them. The legend wrap is especially useful
Great discussion here. Thanks for the comments.
@crayz: I use Gruff regularly in most of my projects but haven’t had time to update it recently. I’ll take a look at that patch (the last time I looked, it still needed some work). I also need to extract some of the less-used graph types into a separate gem so I can update the main core of Gruff more frequently without the extra weight.
Been thinking about this post for the last day. I think there are two related things needed to push Rails to the next level.
(1) A killer app (2) Greater emphasis on speed and scalability
Also a third can’t hurt:
(3) DHH doing a better job of separating his public and private life
A few comments about the above in reverse order. RE: (3) I don’t find DHH’s thoughts about the use of profanity offensive. On the contrary, I find them somewhat stimulating and am glad that he has these sort of opinions. In my view, it is largely this sort of “opinionated” development which distinguishes Rails from its peers. That said, I think DHH should keep his thoughts on such things away from his public blog. Maybe he can move them to an anonymous livejournal or something.
RE: (2) The Rails core committers aren’t doing a good enough job listening to ‘we the people’ and optimizing existing code. Speed is important and no one will use Rails to build a monster app since they would have to rewrite most of the codebase to do so. It is my understanding there are people out there (like the guy from Engineyard, Ezra Z.) who have written their own optimized portions. At the very least we need ways of swapping in and out some of this optimized code.
RE: (1) Let’s be realistic. The creators of the next Youtube wannabe are never going to use Rails in its present state, largely because of the above. Any app that wants to be killer (and by extension, very very large) will use a code base that they can more easily optimize for speed. I’m running into this same problem with my own plan for a social knowledge aggregation site (currently known as the wob-project). Although I’m doing a demo app in Rails, I’m facing the unwelcome prospect of having to switch to another language when I want to scale.
37 Signals can have a bunch of relatively medium-large web apps running but ultimately we all need the emergence of a superstar web app that will really draw people to Rails. I’d very much like to see this happen (and help build it), but at least (2) needs to happen first.
Good site!nnlnl
Nice round-up. Thanks for that!
OK, I’m just a hobbyist, but also a lifelong programmer of 40 years (dating back to punch cards), and I see the RoR thing as a net positive for the evolution of programming, and here’s why …
First of all, if you’re talking about making money, it seems Mr DHH has done very well for himself, what with all the books, conferences, and successful web apps of his that people actually PAY to USE – have any of you done that? Take a lesson people, and go out and try it! You’ll find it’s not all that easy, and you DO need a fair dose of arrogance to believe you can actually take the risk and succeed, instead of just sitting back and taking a paycheck.
Secondly, RoR is a HUGE step forward towards natural language programming; that’s the way Matz designed Ruby, and that’s what DHH has extended. I hope you all realize that in a few years everybody will be able to just talk to their computers and design their own web apps (and more). Rails opens up programming to a whole new crop of people who will create cool new ways to use the Web.
Thirdly: ‘Bigger is not always Better’. Why worry about building some huge thing, when what we should really want, and the future direction of ecosystem-niche reality demands, is many more smaller, focused apps – a ‘Long Tail’ of personalized products is where it’s at!
If DHH seems a little bugged sometimes, it’s probably because he gets frustrated that programmers are so stuck in their esoteric world of cryptic jargon and myopic vision, and that they seem to be standing in the way of progress on these three fronts.