Thursday, June 09, 2011

Quick links

Some of what has caught my attention recently:
  • Oldest example I could find of the "PC is dead" in the press, a New York Times article from 1992. If people keep making this prediction for a few more decades, eventually it might be right. ([1])

  • Amazon CEO Jeff Bezos says to innovate, you have to try many things, fail but keep trying, and be "willing to be misunderstood for long periods of time". ([1])

  • Median tenure at Amazon and Facebook is a year or less (in part due to their massive recent hiring). Also, most people at Facebook have never worked anywhere other than Facebook. ([1])

  • Spooky research out of UW CS and Google that crowdsources surveillance, finding all the Flickr photos from an big event like a concert that happen to include a specific person (no matter at what angle or from what location the crowd of people at the event took the pictures). ([1])

  • You can scan someone's fingerprints from 6 feet away and copy their keys from 200 feet away. ([1] [2])

  • Pretty impressive valuations incubator Y Combinator is getting on its startups: "The combined value of the top 21 companies is $4.7 billion."([1])

  • But even even for some of the more attractive small startups to acquire, those out of Y Combinator, odds of acquisition still are only about 8%, and most of those will be relatively low valuation talent acquisitions. Sometimes it can seem like everyone is getting bought, but it is only a fortunate few who have the right combination of product, team, timing, luck, and network.([1])

  • Someone going solidly for the dumbphone market, which is by far the biggest market still, with a snazzy but simple and mostly dumb phone. That's smart. ([1] [2])

  • Google Scribe makes suggestions for what you are going to type next when you are writing documents. Try starting with "All work and" ([1]).

  • When I started my blog and called it "Geeking with Greg", the word "geek" still had pretty negative connotations, especially in the mainstream. A decade later, things have changed. ([1])

  • Not surprising people don't use privacy tools since the payoff is abstract and the tools require work for the average user to understand and use. What surprises me more is that more people don't use advertising blocking tools like AdBlock. ([1])

  • The sad story of why Google never launched GDrive. ([1])

  • Carriers are going to be upset about Apple's plans to disrupt text messaging. Those overpriced plans are a big business for carriers. ([1])

  • It would be great if Skype acquisition was part of a plan to disrupt the mobile industry by launching a mobile phone that always picks the lowest cost data network (including free WiFi networks) available. Consumers would love that; it could lower their monthly bills by an order of magnitude. ([1] [2])

  • Social data is of limited use in web search because there isn't much data from your friends. Moreover, the best information about what is a good website for you almost certainly comes from people like you who you might not even know, not from the divergent tastes of your small group of friends. As Chris Anderson (author of The Long Tail) said, "No matter who you are, someone you don't know has found the coolest stuff." ([1] [2])

  • Customization (aka active personalization) is too much work. Most people won't do it. If you optimize for the early adopter tinkerer geeks who love twiddling knobs, you're designing a product that the mainstream will never use. ([1])

  • If you launch a feature that just makes your product more complicated and confusing to most customers, you would have been better off doing nothing at all. Success is not launching things, but launching things that help customers. ([1])

  • Google News shifts away from clustering and toward personalization. ([1] [2])

  • Crowdsourcing often works better when unpaid ([1])

  • Eli Pariser is still wrong. ([1])

Monday, June 06, 2011

Continuous profiling at Google

"Google-Wide Profiling: A Continuous Profiling Infrastructure for Data Centers" (PDF) has some fascinating details on how Google does profiling and looks for performance problems.

From the paper:
GWP collects daily profiles from several thousand applications running on thousands of servers .... At any moment, profiling occurs only on a small subset of all machines in the fleet, and event-based sampling is used at the machine level .... The system has been actively profiling nearly all machines at Google for several years.

Application owners won't tolerate latency degradations of more than a few percent .... We measure the event-based profiling overhead ... to ensure the overhead is always less than a few percent. The aggregated profiling overhead is negligible -- less than 0.01 percent.

GWP profiles revealed that the zlib library accounted for nearly 5 percent of all CPU cycles consumed ... [which] motivated an effort to ... evaluate compression alternatives ... Given the Google fleet's scale, a single percent improvement on a core routine could potentially save significant money per year. Unsurprisingly, the new informal metric, "dollar amount per performance change," has become popular among Google engineers.

GWP profiles provide performance insights for cloud applications. Users can see how cloud applications are actually consuming machine resources and how the picture evolves over time ... Infrastructure teams can see the big picture of how their software stacks are being used ... Always-on profiling ... collects a representative sample of ... [performance] over time. Application developers often are surprised ... when browsing GWP results ... [and find problems] they couldn't have easily located without the aggregated GWP results.

Although application developers already mapped major applications to their best [hardware] through manual assignment, we've measured 10 to 15 percent potential improvements in most cases. Similarly ... GWP data ... [can] identify how to colocate multiple applications on a single machine [optimally].
One thing I love about this work is how measurement provided visibility and motivated people. Just by making it easy for everyone to see how much money could be saved by making code changes, engineers started aggressively going after high value optimizations and measuring themselves on "dollar amount per performance change".

For more color on some of the impressive performance work done at Google, please see my earlier post, "Jeff Dean keynote at WSDM 2009".