Saturday, March 20, 2021

Wisdom of the trusted

Flood-the-zone disinformation is a problem for crowdsourced data. Wisdom of the crowds, mass amateurization, and rejection of gatekeepers no longer works with coordinated disinformation campaigns overwhelming rankers, recommenders, and content with shills and spam.

Two decades ago, a lot of us underestimated the negative effects of lower costs for communication and information sharing. While good, it also made propaganda, shilling, and manipulation far easier, and our defenses against disinformation campaigns proved weak.

We in tech were overly idealistic about what would happen as the cost of information and communication dropped. Many thought propaganda would be harder as people could now easily access the truth.

But you can't source reviews from your customers anymore if the vast majority of reviews are paid shills. You can't rank using usage data if ratings and clicks are mostly fake.

Crowdsourced information, including web crawls, reviews, and commentary, only works when almost everyone is independent and unbiased. Coordinated disinformation breaks crowdsourcing.

Flood-the-zone shouldn't have been a surprise, but it was. Propaganda and manipulation are winning because we still treat inauthentic behavior as real.

While there is plenty of mostly-deserved love for big data, often less is more when you live in an adversarial, flood-the-zone world. Wisdom of the crowds has an assumption of independence between agents, which now has been broken by coordinated disinformation campaigns.

If you are looking at garbage, there is no information. Adding disinformation to good data purely makes things worse. It's like making a milkshake, then eying a huge putrid sack of night soil nearby. Sure, you could add some of that to what you made, but even a little is going to make it worse. If there is crap everywhere, you might want to stick with what you can prove to be good.

Polling, one of the oldest forms of crowdsourced information, has been impacted too. The trend in recent years is that low response rates and shilling make it so expensive to poll that Pew Research gets better data cheaper by forming and managing a large paid panel of trusted experts.

For those working in machine learning, for those trying to work with big data, reputation and reputable sources have to be the response in a flood-the-zone world. When most of the data is bad, how you filter your data becomes the most important thing.

We have a big challenge ahead, countering disinformation using reputation and lack of reputation. In a flood-the-zone world, most data out there is now bad to useless. Isolating the useful requires skepticism toward data, like TrustRank, starting untrusted, bad until proven good.

Reviews should discard anything even resembling a shill, giving visibility only to reviews from independent and trustworthy customers. Recommender systems and rankers should focus on the data from and related to proven sources, and weight anything unknown as questionable at best and likely worthless. Most crowdsourced data for machine learning, from clicks to content, is going to have to be viewed with skepticism.

Inauthentic behavior and coordinated disinformation campaigns have shilled wisdom of the crowd to death. For reliable big data in a flood-the-zone world, it will have to be wisdom of the trusted.

Tuesday, December 15, 2020

When will virtual reality take off? The $100 bet.

About four years ago, Professor Daniel Lemire and I made a $100 bet on how quickly virtual reality would reach a broad, mainstream market. Specifically, my side of the bet was, "Virtual reality hardware (not counting cardboard) will not sell more than 10M units/year worldwide before March 2019." He bet that it would.

In early 2020, we decided to wait settle the bet because it looked like there was some chance VR would reach 10M units/year in 2020. Because of COVID and people looking for entertainment at home, Valve's release of Half Life Alyx, Supernatural (the VR exercise program), and big pushes on consumer VR by several companies, we wanted to wait and see if it was off by just one year, if 2020 was the year.

At this point, the results are in, and it is clear VR has not reached far beyond early adopters and enthusiasts. Estimates of total hardware sales vary depending on what is considered VR hardware, but most estimates I've seen have worldwide unit sales at around 5-6M in 2020.

Barron's has a nice summary: "We’ve been talking about virtual reality for decades, but it’s gone pretty much nowhere. Despite all of our advances in tech, VR hasn’t been able to bridge the physical and digital realms in any substantial way." TechCrunch adds, "There are signs of growth though it’s clear [VR] is still a niche product."

So what went wrong? Looking back at VR hype in 2016, there were a lot of reasons to be optimistic: HoloLens from Microsoft, Sony entering VR with Playstation VR, Valve pushing hard on VR in the Steam store and with their own products, Xbox looking like it might do VR, Google showing interest in VR, and, though it always seemed like vaporware to me, there was a lot of excitement around the promises made by heavily-funded MagicLeap. It looked likely that someone would make a must-have game or other compelling use of VR that might attract tens of millions of people.

Speculating a bit, I think the issue here goes beyond just needing more time, so beyond waiting for gradual acceptance of VR and growth. I think the problem is that the non-virtual-reality experience is close enough for most purposes, making VR uncompelling to set up and use.

For example, take the virtual tourism experience of visiting the International Space Station in Google Earth. It's fun and compelling enough without virtual reality, so VR in virtual tourism only a little bit of wow to the experience. Half Life Alyx seems to me to suffer from the same problem, a fun game with some compelling content, so great to try, but not a must-have. Exercise programs like Supernatural or Beat Saber fall in the same category, fun, cool to try, but not something without okay substitutes or alternatives.

At the time we made the bet back in 2016, I said something similar about why I might lose the bet: "There are several wild cards here. For example, it is possible that much cheaper units can be made to work. It's possible that someone discovers very carefully chosen environments and software tricks fool the brain into fully accepting the virtual reality, especially for gaming, increasing the appeal and making it a must-have experience for a lot of people. As unsavory as it is, pornography is often a wild card with new technology, potentially driving adoption in ways that can determine winners and losers. A breakthrough in display (such as retinal displays) might allow virtual reality hardware that is much cheaper and lighter. Business use is another unknown where virtual reality could provide a large cost savings over physical presence. I do think there are many ways in which I could lose this bet."

Unfortunately, I don't think such must-have, compelling VR experiences exist. Perhaps at some point it will. Chris Pruett, who runs part of Oculus, speculated about that, saying: "My guess would be something that is highly immersive, that involves active motion of your body, and ... it's probably going to be something that you either play with other people or is shareable with other people." That sounds plausible to me, though, more broadly, I think it has to be a must-have experience without okay substitutes in non-VR, which is a high bar. My prediction now in 2020 would be that VR will continue to struggle for years to break out beyond enthusiasts and early adopters, at least until it has a truly must-have experience.

I think Daniel Lemire took the harder side of this bet, so I'll match his $100 donation to Wikipedia to settle the bet. Back in 2016, I did add a couple ways of making my side of the bet even harder, saying I doubted even over three years in 2016-2019 that VR would sell a total of more than 10M/units, which appears to be close, and that Google Cardboard-like devices wouldn't go beyond being just a toy, so not regularly used by tens of millions, which looks like it was correct.

And I want to thank Daniel for making this bet. Whether you are betting with the hype or against it, along with conventional wisdom or against the flow, it's hard to publicly take a stand and one way or another and be willing to be wrong, especially when big company money is betting against you. This was an interesting bet.

If you enjoyed this, you might also be interested in our 2012 bet about whether tablets will replace PCs.

Update: Daniel Lemire has a post up on his thoughts on the bet, "Virtual reality… millions but not tens of millions… yet".

Friday, December 04, 2020

Facebook and investing in the long-term

Kevin Roose, Mike Isaac and Sheera Frenkel at the New York Times had a great piece ([1] [2]) on the internal debate inside Facebook on removing disinformation:
Facebook engineers and data scientists posted the results of a series of experiments called "P(Bad for the World)." ... The team trained a machine-learning algorithm to predict posts that users would consider "bad for the world" and demote them in news feeds. In early tests, the new algorithm successfully reduced the visibility of objectionable content.

But it also lowered the number of times users opened Facebook, an internal metric known as "sessions" that executives monitor closely.

Another product, an algorithm to classify and demote "hate bait" — posts that don’t strictly violate Facebook’s hate speech rules, but that provoke a flood of hateful comments ... [Another] called "correct the record," would have retroactively notified users that they had interacted with false news and directed them to an independent fact-check ... [Both were] vetoed by policy executives who feared it would disproportionately show notifications to people who shared false news from right-wing websites.

Many rank-and-file workers and some executives ... want to do more to limit misinformation and polarizing content. [Others] fear those measures could hurt Facebook’s growth, or provoke a political backlash ... Some disillusioned employees have quit, saying they could no longer stomach working for a company whose products they considered harmful.
The article is an insightful look at the struggle inside Facebook on recommender systems for news, metrics, and short vs. long-term metrics and growth. Key is fear of harming short-term metrics like sessions per user and engagement.

Any attempt to increase quality of news or ads is going to result in a short-term reduction in metrics engagement, usage, and revenue. That's obvious and not the question to ask. The question to ask is, does it pay off in the long-term?

It's unsurprising that once you've kicked off all users who hate what Facebook has become and addicted the rest to clickbait, the remainder will use Facebook less in the short-term if you improve the quality of content.

This is just like any other investment. If you invest in any large expense, you expect your short-term profits to drop, but you're betting that your long-term profits will rise. In this case, increased news quality is an investment in bringing back lapsed users.

Even measured over weeks, sessions per user is going to take a hit with a change to news quality because users who like higher quality news already disengaged and abandoned and current heavy users won't like the change. It will take a long time to pay off.

For Facebook, reducing disinformation probably would also be an investment in other areas. Facebook is polluting society with disinformation, externalizing costs; cutting disinformation is an investment in reducing regulation risk from governments. And Facebook wants good people, and many good people are leaving ([1]) or won't even consider working there because of their practices, a considerable long-term cost on the company; cutting disinformation is an investment in recruiting and retention. So Facebook probably would see benefits beyond lapsed users.

Facebook and others need to think of reducing disinformation as an investment in the future. Eliminating scams, low quality ads, clickbait, and disinformation often will reduce short-term metrics, but is a long-term investment in quality to reduce abandons, bring back lapsed users, and in other long-term business goals. These investments take a long-time to pay off, but that's why you make investments, for the long-term payoff.

Tuesday, November 26, 2019

Papers and posting

If you haven't seen it, Adrian Colyer's excellent blog has great reviews and summaries of recent papers. Back when this blog started in 2004, there weren't many people summarizing research papers. Many more are now, which is part of why I post less now. Adrian's blog is excellent and similar to what I used to do, but I think better in many ways. You can also follow Adrian Colyer on Twitter.

While I'm talking about summarizing papers, I want to highlight two lines of work that had an impact on me in the last few years and that I think deserve much more attention. Both argue we, as in all of us in tech, are doing something important wrong.

The first argues that our metrics usually are off, specifically way too focused on short-term goals like immediate revenue. This is the work started by the fantastic Focus on the Long-term out of Google and continuing from there (including [1] [2]). Because much of what we all do is an optimization process -- ML, deep learning, recommendations, A/B testing, search, and advertising -- having the targets wrong means we are optimizing for the wrong thing.

Optimizing for the wrong thing is ubiquitous in our industry. It may, for example, cause almost everyone to show too many ads and too many low quality ads. If everyone has their metrics subtly wrong, everything we make, and especially everything in the ML community, may be aiming for the wrong target.

The second is Kate Starbird's work on disinformation campaigns. Across many recent papers, Kate argues that the traditional classifier approach to spam, trolls, and shills has been failing. Adversaries can create many accounts and enlist real humans in their disinformation effort. Knocking a few accounts away does nothing; it is like shooting bullets into a wave. Instead, it is important to look at the goals of disinformation campaigns and make them more expensive to achieve. Because shills impact so many things we do -- training data for ML and deep learning, social media, reviews, recommendations, A/B testing, search, advertising -- our failure to deal with shills means the assumptions all of these systems have about the data all being equally good are wrong, and the quality of all these systems is reduced.

Solutions are hard. I'm afraid Kate's advice on solutions is limited. But I would say solutions include whitelisting (using only experts, verified real people, or accounts that are expensive to create), recognizing likely disinformation as it starts to propagate and slowing it, and countering likely disinformation with accurate information where it appears. Those replace outdated whack-a-mole account classifiers and work across multiple accounts to counter modern disinformation campaigns. Manipulation and shilling from sophisticated adversaries is ubiquitous in our industry. Until we fix this, many of our systems produce lower quality results.

Finally, I am posting a lot less here now, so let me point to other resources for anyone who liked this blog. I still post frequently on Twitter; you can follow me there. On AI/ML, there's a lot of great writing by a lot of people, far too many to list, but I can at least list my favorites, which are Fran├žois Chollet and Ben Hamner on Twitter. On economics and econometrics, which I enjoy for adding breadth to AI/ML, my favorites are economists Noah Smith and Dina Pomeranz on Twitter.

Wednesday, May 08, 2019

Tech and tech idealism

It's been almost 2 years since my last post! I don't know if anyone is still reading this. If you are, thank you!

Why haven't I posted more? Partly it is the broad transition to microblogging, which everyone is using more than long form. But part also is that I have negative feelings about where tech has been going.

I'm a tech idealist. I think tech can and should be a force for good in the world. I have spent most of my life trying to build systems where computers are helping humans. Sometimes this is by computers sifting information that is hard for people to find on their own. Sometimes this is by computers surfacing other people that can help.

Lately, some tech companies have been favoring exploitation and deception. Data is being used to manipulate. Tech is becoming customer hostile.

I've been lucky. I have gotten to work on some amazing things. There is a joy to helping someone discover a new book they will love, a bit of knowledge added to a life. Many people feel overwhelmed by the news and information in their lives, and sorting through to find what is truly important is too hard. Ads shouldn't be so annoying and irrelevant, and, you know what, they don't have to be. I've enjoyed helping people find and discover whatever they need online.

But looking at where we are in tech now, it feels like a dot com bubble again. Get rich quick. It's not building something that people love, but get the buck. Greed feeds short-term thinking. Grab that next bonus and get out before the wreckage hits.

Tech idealism is still out there. There still are many people building things that help people. There is research, the creation of knowledge and new ways to help even more people. There are many people using computers and data for good.

And there are many new people getting into computer science, which is fantastic. Computers are a force multiplier. Computers make people more productive and more powerful. Computer science and data science are just starting to have an impact in other fields.

The interdisciplinary opportunities are everywhere and exciting. We know almost nothing about our own oceans; there are huge opportunities for discoveries in biology from undersea probes and drones. We are just starting to image the entire night sky frequently, and sifting through that data with massive computing power will forever change astronomy. The field of economics is shifting to data and behavior over theory. Archeology can be fueled by processing massive amounts of satellite imagery. In field after field, computers and data are making the once impossible possible.

Tech idealism is coming back. Something may have to come to flush away some of those just seeking quick profits. Some of the worst abuses may have to be obvious failures before they are rained in. But it will change.

Computers and data are a force multiplier, allowing people to do more than they could before. Working at massive scale, computers help us understand and discover. In long-term, tech is a force for good.

Saturday, June 24, 2017

Two decades of recommendations

IEEE Internet Computing just celebrated its 20th anniversary.

On its 20th anniversary, the editorial board created its first ever “The Test of Time” award. I'm honored to say they gave it to our 2003 article, " Recommendations: Item-to-Item Collaborative Filtering", which continues to be accessed, cited, and used in industry and research many years after its original publication.

In addition, for the 20th anniversary issue of IEEE Internet Computing, we wrote a new article, “Two Decades of Recommender Systems at". Some excerpts:
For two decades now, has been building a store for every customer. Each person who comes to sees it differently ... It's as if you walked into a store and the shelves started rearranging themselves, with what you might want moving to the front, and what you're unlikely to be interested in shuffling further away. launched item-based collaborative filtering in 1998, enabling recommendations at a previously unseen scale for millions of customers and a catalog of millions of items. Since we wrote about the algorithm in IEEE Internet Computing in 2003, it has seen widespread use across the Web, including YouTube, Netflix, and many others.

The algorithm's success has been from its simplicity, scalability, and often surprising and useful recommendations, as well as desirable properties such as updating immediately based on new information about a customer and being able to explain why it recommended something in a way that's easily understandable.

What was described in our 2003 IEEE Internet Computing article has faced many challenges and seen much development over the years ... We describe some of the updates, improvements, and adaptations for item-based collaborative filtering, and offer our view on what the future holds for collaborative filtering, recommender systems, and personalization.


What does the future hold for recommendations? ... Discovery should be like talking with a friend who knows you, knows what you like, works with you at every step, and anticipates your needs.

Recommendations and personalization live in the sea of data we all create as we move through the world, including what we find, what we discover, and what we love ... Intelligent computer algorithms leveraging collective human intelligence ... Computers helping people help other people.

The field remains wide open. An experience for every customer ... offering surprise and delight ... is a vision none have fully realized. Much opportunity remains to add intelligence and personalization to every part of every system, creating experiences that seem like a friend that knows you, what you like, and what others like, and understands what options are out there for you.

Sunday, June 11, 2017

Quick links

Some of the tech news I found interesting lately, and you might too:
  • Jeff Bezos: "Many decisions are reversible, two-way doors. Those decisions can use a light-weight process. For those, so what if you’re wrong? .... If you’re good at course correcting, being wrong may be less costly than you think, whereas being slow is going to be expensive for sure." ([1])

  • Jeff Bezos: "I would say, a lot of the value that we’re getting from machine learning is actually happening beneath the surface. It is things like improved search results. Improved product recommendations for customers. Improved forecasting for inventory management. Literally hundreds of other things beneath the surface." ([1])

  • A good summary of Mary Meeker's 2017 report. A key highlight is saturation in smartphones and internet usage. ([1])

  • New Google AI incubator: "Investment arm aimed squarely on artificial intelligence ... will operate almost like an incubator with a shared workspace for AI startups and mentorship" ([1] [2])

  • Lots of good labeled data (reliable ground truth) is the key to success with AI ([1] [2] [3] [4])

  • AI in the real world is a lot harder than ideal conditions in part because you see crazy things like robots getting attacked by humans ([1] [2])

  • "The Google [Chrome] ad-blocker will block all advertising on sites that have a certain number of 'unacceptable ads,' according to The Wall Street Journal. That includes ads that have pop-ups, auto-playing video, and 'prestitial' count-down ads that delay the display of content." ([1])

  • Nice ACM Queue article from Google SREs on availability as a combination of subservice reliability, rapid recovery, and setting expectations ([1])

  • "Designing a [software] library to reduce cognitive load is still the exception, not the rule" ([1] [2])

  • A lesson for bigger companies, investing in the long-term with your researchers, who are often working a few years ahead of what you'll need now ([1])

  • Wow: "The Melt’s blundering trajectory is instructive ... Entrepreneurs frequently embark on these missions with vast sums of money and a deep belief in technology’s power to solve all problems — which is not always a formula for success .... They were all good people, and they all wanted good things. They just didn’t know anything about running restaurants." ([1])

  • "The once-hot social network was built on the idea that people would enjoy having anonymous conversations with people close by. That’s a fantastic concept until you remember that anonymous internet person and by definition near you are scary as hell in practice." ([1])

  • Great teardown of the Juicero, includes some excellent business advice on iterative development and testing your ideas on real customers ([1] [2])

  • "When the US government discovers a vulnerability ... it can keep it secret and use it offensively ... or it can alert the software vendor and see that the vulnerability is patched, protecting the country ... Every offensive weapon is a (potential) chink in our defense." ([1])

  • On spearfishing attacks: "By a careful design and timing of a message, it should be possible to make virtually any person click" ([1] [2])

  • Schneier on forging voices: "I don't think we're ready for this. We use people's voices to authenticate them all the time, in all sorts of different ways." ([1])

  • Facebook says, "We have had to expand our security focus... to include more subtle and insidious forms of misuse, including attempts to manipulate civic discourse and deceive people" ([1] [2])

  • Remarkable and concerning that this is possible: "By accessing accelerometer and gyroscope sensors, the Web-hosted JavaScript measures subtle changes in a phone's angle, rotation, movement speed, and similar characteristics. The data, in turn, can reveal sensitive information about the phone and its user ... [including] the keystrokes being entered" ([1])

  • Nice high level description here of the difference between what Apple and Google are doing for privacy-preserving machine learning. In brief, Apple adds noise to the data to preserve privacy, but Google learns on the device then sends the updates to the machine learned models back (much like parameters servers in deep learning). The truth is they're probably both doing both, but it's still a good thing to think about. ([1])

  • Using battery backup to optimize gas power plants by being able to skip the expensive bits for gas turbines, sitting in standby because of lengthy startup times. It's easy and practical, a nice example of low hanging fruit with major impact. ([1])

  • Good data on the projected costs of energy sources ([1])

  • Good data on the newspaper industry. There's a curious spike in ad revenue from 1980-2000 that isn't matched by subscriptions. ([1])

  • Jeff Bezos is making journalism profitable: "The Post has said that it was profitable last year — and not through cost-cutting ... The Post has gone on a hiring spree. It has hired hundreds of reporters and editors and has more than tripled its technology staff ... third straight year of double-digit revenue growth ... 'You have to be great at technology. You have to be great at monetization. But one thing I think we’re proving is that if you are, great journalism can be profitable.'" ([1])

  • How Google took over the classroom, great article, but misses that the failure of iPads was a big piece of this ([1] [2])

  • Duolingo's excellent efforts to help people learn English, which can be a tool for economic or educational advancement ([1])

  • Amazon Web Services cuts prices again, remarkable ([1])

  • Almost all cloud workloads right now are not cloud optimized, so the customers mostly moved a system built for fixed hardware resources to the cloud and then run idle a lot rather than redesigning to optimize with dynamically scaling ([1])

  • Latest version of Google Earth is impressive, definitely worth trying ([1])

  • Brent Smith and I received the first ever IEEE Internet Computing Test of Time award for our 2003 paper on Amazon's recommender system. In a new article for the IEEE Internet Computing 20th Anniversary Issue, we look back at the last two decades. ([1])

  • A virtual reality game that succeeds at taking advantage of what it can do well and what it can't to create a fully immersive experience ([1])

  • Somehow, I missed that Chris Sacca is retiring. Amazing career and influence he had, and impressive to decide to go an entire new direction now. ([1])

  • In a Stack Overflow survey, what software engineers care about, it's who they work with, what they are doing, and what they learn far more than salary. In the top five items, three are about who you work with and what you learn, one is benefits, and one is commute. But the benefits are complicated -- it's not salary, stock, and bonus -- but the top items all things related to work environment and commute, vacation, and health care. ([1])

  • Great interview with the CEO of Coursera: "Humility and the ability to listen well are the big things I look for ... If you want to understand people, you need to hear them ... [Also have] ambitious goals to lift the organization up and everybody with it. Setting goals that are ambitious but also achievable is an important skill." ([1])

  • Great quote from Jeff: "At Amazon, we've had a lot of inventions that we were very excited about, and customers didn't care at all. And believe me, those inventions were not disruptive in any way. The only thing that's disruptive is customer adoption." ([1])

  • Nice line in Dan Ariely's book Payoff: "If you really want to demotivate people, shredding their work is the way to go, but ... you can get almost all the way there simply by ignoring their efforts." ([1])

  • Xkcd points out minor changes in methodology yield radical changes in data visualizations of most unusually popular activity in a location ([1])

  • Xkcd on machine learning, disturbingly close to reality ([1])

  • Xkcd on hard problems ([1])

  • Xkcd on survivorship bias ([1])

  • Xkcd on unhelpful code reviews ([1])

  • Very funny that Burger King ran an ad with "OK, Google" and it works. Once again Xkcd was hilariously prescient about this. ([1] [2])

  • SMBC comic on bayesian inference: "Given his low priors..." ([1])

  • SMBC comic: "Then it occurred to me, hey, I've got like a sample size of one here, and it's not double blind." ([1])

  • SMBC comic on behavioral economics ([1])

  • SMBC comic: "Wait, are you going to turn my life's work into a joke about butts or something?" [1])

Sunday, April 30, 2017

All Crunchzilla tutorials now open source

All the code is now available for all the Crunchzilla coding tutorials.

Code Monster, Code Maven, and Game Maven from Crunchzilla have been used by hundreds of thousands of people around the world to experiment with learning to write computer programs.

There have been many requests to make them and available in languages other than English.

By open sourcing the Crunchzilla tutorials, I hope three things might happen:

Translations: I hope others are able to take the content and translate part or all of it into languages other than English for use in more classrooms around the world.

New lessons: New tutorials might teach programming games, working through puzzles or math problems, or perhaps a more traditional computer science curriculum aligned with a particular lesson plan.

Entirely new tutorials: Some of the ideas and techniques -- including the step-by-step learn-by-doing style, live code, informative error messages, and avoiding infinite loops in students' code -- might be useful for others.
The code was designed to be all static, so you can easily create your own version just by editing the files and then putting all the files together on your own server. There is a single JSON file that contains all the lesson content.

If you use the code for anything that helps children learn, I'd love to hear about it (please e-mail me at

Sunday, April 02, 2017

Quick links

A carefully picked list of some of the tech news I enjoyed recently:
  • So, you know that prototype we showed you? Turns out AI in real world conditions is hard. ([1] [2] [3])

  • Artificial intelligence expert Yann LeCun says, "There have been, on the face of it, impressive demonstrations, [but] those are not as impressive as they look ... They don't have common sense ... One of the things we really want to do is get machines to acquire the very large number of facts that represent the constraints of the real world just by observing it through video or other channels. That’s what would allow them to acquire common sense, in the end." ([1])

  • Genetic algorithms and neural networks are back. It feels like the 1990s all over again. ([1])

  • Bringing more novices to AI now is the way to get more experts and advances later ([1])

  • Nice results from focusing on errors that matter to people, the perceived quality of the system by humans, not theoretical accuracy ([1] [2])

  • Success often comes from trying many things: "Start ... with a hazy intuition or vision ... After a lot of trial and error they get closer and closer to discovering what their idea is ... Seeking novelty instead of objectives is risky — not every interesting thread will pay off — but ... the potential payoffs are higher." ([1])

  • Research includes people able to do things no one else can, including having data or compute at the frontier beyond what anyone else has done before ([1] [2])

  • 6.3M virtual reality headsets sold in 2016, but almost all so far just the cheap toys where you slot your smartphone in to use as the screen ([1] [2])

  • "Total [tablet] sales sinking 15.6%, year on year, with sales of 174.8M units in 2016 compared to 2015's 207.2M" ([1])

  • For the first time, more people in the US using Netflix than a DVR: "54 percent of US adults reporting they have Netflix in their households compared to the 53 percent of US adults that have DVR" ([1])

  • The Economist: "Amazon’s heady valuation resembles a self-fulfilling prophecy. The company will be able to keep spending, and its spending will keep making it more powerful" ([1])

  • "What has surprised AWS as the cloud has evolved ... I don’t think in our wildest dreams we ever thought we’d have a six- to seven-year head start" ([1])

  • ... and that is true in retail for Amazon as well ([1] [2] [3])

  • "Yahoo is perhaps most famous for destroying all of its best social properties. From its hideous deformations of Flickr and neglect of Upcoming to its starvation of Delicious and torment of GeoCities users, the company excelled at buying great things and turning them into unusable parodies of themselves. Execs seemed to profoundly misunderstand why people used the sites they bought." ([1])

  • "Google will account for 78 percent of search ad revenue in 2017, while Facebook will get 39 percent of display ad revenue. Everyone else ... is fighting over the scraps." ([1])

  • Culture is created by what you publicly reward, not what you say ([1] [2] [3])

  • "The problem with bad processes is that they institutionalize inefficiency. They ensure that things will be done the wrong way, over and over and over again" ([1] [2])

  • "Burnout begins when a worker feels overwhelmed for a sustained period of time, then apathetic and ultimately numb .... Workers who used to take the lead on projects grow taciturn during meetings. Top performers start coming in late, leaving early and watch their careers stall ... Burnout is claiming victims at work, and companies aren’t ready to cope" ([1])

  • A lot of companies have merely medium data, not big data: "Hundreds of enterprises were hugely disappointed by their useless 2 to 10TB Hadoop clusters ... Their data works better in other technologies" ([1])

  • Lack of incentives leads to poor Internet of Things security ([1])

  • As Javascript ages, it repeats many of the problems of the past: "Using data from over 133K websites, we show that 37% of them include at least one library with a known vulnerability" ([1])

  • "What are some things you wish you knew when you started programming?" ([1] [2])

  • Many Xkcd comics are both funny and prescient, and this one on encryption seems particularly relevant right now ([1])

  • Xkcd comic on friends that have an Amazon Echo ([1])

  • SMBC comic on "existential sort". Don't miss the hovertext: "Also, any list can be immediately sorted by just pretty much being fine with it the way it is." ([1])

Saturday, April 01, 2017

Book review: Radical Candor

This just came out, the book Radical Candor by Kim Scott. It's a good read on managing and focused on people. I'd recommend it if you are a manager or help others manage people.

I'd summarize it by saying it takes a teaching and mentoring approach to management, very much of the school that managers primarily exist to help the people on their team. The advice is both practical and actionable, with specific advice for running 1:1s and meetings, and focused how to encourage conversations where people strive to improve themselves as well as helping others.

Some carefully selected quotes from the book:

"It seems obvious that good bosses must care personally about the people who report directly to them ... And yet ... "

"It turns out that when people trust you and believe you care about them, they are much more likely to accept and act on your praise and criticism, tell you what they really think about what you are doing well and, more importantly, not doing so well, engage in this same behavior with one another ... embrace their role on the team, and focus on getting results"

"When you're the boss, it's awkward to ask your direct reports to tell you frankly what they think of your performance, even more awkward for them than it is for you. To help, I [ask] ... 'Is there anything I could do or stop doing that would make it easier to work with me?' ... It is essential that you ... commit to sticking with the conversation until you have a genuine response. One technique is to count to six before saying anything else, forcing them to endure the silence. The goal is not to bully but to insist on a candid discussion ... Then listen with the intent to understand ... Once you've asked your question and embraced the discomfort and understood the criticism, you have to follow up by showing that you welcome it. You have to reward the candor if you want to get more of it ... Make a chance as soon as possible ... show you're trying."

"If you can absorb the blows, the members of your team are more likely to be good bosses to their employees when they have them ... The rewards of watching people you care about flourish and then help others flourish."

"The ultimate goal of Radical Candor is to achieve results collaboratively that you could never achieve individually ... A culture of guidance ... An exemplary team ... self-correcting quality whereby most problems are solved before you are even aware of them ... Don't start by bossing people. They'll just hate you. Start by listening to them."

Sunday, February 26, 2017

More quick links

Some of the tech news I found interesting lately, and you might too:
  • "In addition to making our systems more intelligent, we have to make them more intelligible too ... AI systems to augment human capabilities ... A human-centered approach is more important than ever." ([1])

  • "Understanding the brain is a fascinating problem but ... separate from the goal of AI which is solving problems ... We don’t need to duplicate humans ... We want humans and machines to partner and do something that they cannot do on their own." ([1])

  • "Machine learning and reasoning to help doctors to understand patient outcomes -- in advance of poor outcomes ... a great deal of low-hanging fruit where even today’s AI technologies are well positioned to help ... error detection, alerting, and decision support ... could save hundreds of thousands of lives per year" ([1] [2])

  • "Google's first entirely on-device ML technology ... machine intelligence ... run on your personal phone or smartwatch" ([1])

  • Accelerometers and heart rate monitors in earbuds, clever and avoids the need for a separate wearable ([1])

  • On Google's business: "Mobile search and YouTube were the main drivers of Google’s strong performance ... Google’s market share ... is above 90 percent on mobile devices" ([1] [2] [3])

  • "AI is the next platform for Facebook right now. The company is quietly approaching this initiative with the same urgency as its previous Web-to-mobile pivot." ([1])

  • "Microsoft formed a new 5,000-person engineering and research team to focus on artificial intelligence products" ([1])

  • Qi Lu leaves Microsoft for Baidu, and Jan Pedersen leaves Microsoft for Twitter. ([1] [2])

  • Not sure how well known this is: "Facebook collects information about pages [you] visit that contain Facebook sharing buttons ... And in case that wasn’t enough, Facebook also buys data about its users’ mortgages, car ownership and shopping habits from some of the biggest commercial data brokers. Facebook uses all this data to offer marketers a chance to target ads to increasingly specific groups of people. Indeed, we found Facebook offers advertisers more than 1,300 categories for ad targeting — everything from people whose property size is less than .26 acres to households with exactly seven credit cards." ([1])

  • Interesting example for the news industry: "Doubling down on traditional journalism and investing heavily in new ways to deliver it, through smartphone apps, voice-activated speakers and e-readers. The Post’s digital effort has become the envy of the industry, with as many as 80 software engineers, developers and others working alongside reporters and editors to present the news in real time." ([1])

  • "Bezos has worked to create a culture at Amazon that’s hospitable to experimentation ... developing products customers will actually want to pay for ... experiments start small and grow over time ... a small team to experiment with the idea and find out if it’s viable ... if a team succeeds in smaller challenges, it’s given more resources and a larger challenge to tackle ... prioritize launching early over everything else ... learn as quickly as possible whether an idea that sounds good on paper is actually a good idea in the real world ... getting a product into the hands of paying customers as quickly as possible and taking their feedback seriously ... avoids wasting years working on products that don’t serve the needs of real customers." ([1])

  • New direction for the cloud, just small pieces of code running somewhere (you don't care where) and data stored somewhere (you don't care where), all auto scaled ([1] [2])

  • "Many failed ideas have been resuscitated and rebranded as successful products and services, owned and managed by people other than their originators. Behind almost every popular app or website today lie numerous shadow versions that have been sloughed away by time. Yet recognition of the group nature of the enterprise would undermine a myth that legitimizes the consolidation of profit, for the most part, among a small group of people." ([1])

  • For those of us tracking virtual reality: "While Facebook does not provide sales figures for the $599 Oculus Rift headset ... analysts believe they are slow. One research firm ... estimated the company sold only about 355,000 by the end of last year." ([1] [2] [3])

  • A surprising level of detail here on what software development is like inside of Google. I agree with most of it, and highly recommend reading at least Section 2. ([1] [2])

  • Great blog post summarizing NIPS 2016. Highlights are what wins Kaggle competitions, why deep learning works, latest twiddles to deep learning and reinforcement learning, why dialogs (chat) still doesn't work, and that Baidu has products who's only value is in the data they collect (not direct revenue, just the explore part of explore/exploit, learning how to be more effective). ([1])

  • Ease of use is badly underrated: "Using TensorFlow makes me feel like I’m not smart enough to use TensorFlow; whereas using Keras makes me feel like neural networks are easier than I realized." ([1])

  • New paper by Geoff Hinton and Jeff Dean, essentially a very large ensemble of neural networks with sparsity enforced to minimize the computational cost ([1])

  • Thoughtful comments on engineering management ([1])

  • Different people we work with in tech tend to have different ideas of what it means to get things done ([1])

  • "People with different backgrounds bring new information. Simply interacting with individuals who are different forces group members to prepare better, to anticipate alternative viewpoints and to expect that reaching consensus will take effort." ([1])

  • Meetings are expensive -- a 10 person meeting for an hour costs a few thousand dollars -- and people hate meetings too. Some good reoccurring themes here are to keep meetings small, short, write a tight agenda ahead of time, stay off your laptop and phone, and try to finish early. ([1])

  • Disappointing game theory tidbit of the day, the Joy of Destruction game shows people enjoy causing harm when they can do it without consequences ([1] [2])

  • Great data visualizations from 538, not just eye candy but convey information quickly ([1])

  • "Tesla has 1.3 billion miles of car-driving data thanks to its Autopilot-equipped vehicles that are already on the road before competitors in Detroit and Silicon Valley can roll self-driving cars off the lot. It’s a massive competitive advantage." ([1])

  • Fun details on laying undersea internet cables from Amazon Web Services Distinguished Engineer James Hamilton ([1])

  • "All future wars will begin as cyberwars" ([1])

  • Impressive plans from China's space program, probes on the far side of the moon and on Mars in the next four years ([1])

  • For those interested in education, MIT's popular and excellent Scratch has published a dataset of how people learn computational thinking ([1])

  • What has achieved is very impressive: "Trained 50,000 new K-12 computer science teachers ... More than 20 million lines of code have been written by ... more than one million K-12 students ... we expect to dramatically change the demographics of AP Computer Science this year" ([1])

  • Funny article from The Onion on having too many browser tabs open ([1])

  • SMBC comic on the universe as A/B testing ([1])

  • SMBC comic on behavioral economics and anchoring ([1])

  • SMBC comic: "The wise man was put to death in the most mathematically insulting way possible" ([1])

  • Xkcd comic on what phones are, random emotional stimuli to replace boredom with anxiety ([1])

  • Xkcd comic on being an overoptimizer ([1])

Sunday, December 18, 2016

Quick links

Some of the tech news that caught my attention lately:
  • Humans working for the AI: How we get ground truth for machine learning ([1])

  • Deep learning helping on diagnostic medical imaging with accuracy at human level ([1] [2] [3] [4])

  • BHAG from Intel: "Intel aims to deliver up to 100x reduction in the time to train a deep learning model over the next three years compared to GPU" ([1])

  • Deep learning's success is mostly a lot of data paired with an algorithm that can take advantage of a lot of data ([1])

  • Fun! "A software platform for evaluating and training intelligent agents across the world’s supply of games, websites and other applications ... Agents use the same senses and controls as humans: seeing pixels and using a keyboard and mouse." ([1])

  • Details on Duolingo's learning algorithms, including that they found what worked best for students using A/B tests ([1])

  • A rant on hype-driven development ([1] [2])

  • Building finished products is hard ([1])

  • "There is an optimal newness for ideas -- advanced yet acceptable" ([1])

  • Massive expansion of Facebook in Seattle. Seattle is increasingly becoming a mini Silicon Valley ([1] [2])

  • Andy Jassy optimistic Amazon Web Services will become a $100B business ([1])

  • Detailed comparison of pricing on Google, AWS, and Azure. To summarize, it's complicated, and what your cheapest option will be depends a lot on what you're doing. ([1])

  • "Google has been carbon neutral since 2007, and [in 2017] we'll be powered by 100% renewable energy as our newest wind and solar farms come online" ([1])

  • Likely to see truly massive wind turbines in the near future ([1])

  • "The Waffle House Index also stands for something less obvious. It is an indicator of how complex and long supply chains are — for food, for fuel, for power — and of what it takes to plan around infrastructure that can be fragile in unexpected ways." ([1])

  • Xkcd: "Of course, 'Number of times I've gotten to make a decision twice to know for sure how it would have turned out' is still at 0." ([1])

  • "Not one, nor two, but five major VC funds reached out about investing in Rocket AI ... The ultimate fake AI company ... AI is at peak hype, and everyone in the community knows it." ([1])

Saturday, December 10, 2016

Book review: Chaos Monkeys

Cynical, mercenary, and dark, this book aptly serves as an opposing view for any idealism you may have been feeling about Silicon Valley startups or their bigger brethren. Some of us work in technology to make a difference. That is not what you will find in this book.

It is a tale of a startup that wasn't really a startup, three people with no real product acquired after 10 months. It is a tale of sales and personal marketing, spinning unfavorable realities into golden-sounding tales capable of jumping the next hurdle and moving on to the next deal. It is a tale of greed and personal ambition, everything viewed through a Wall Street lens of climbing a hierarchy of wealth and power, some in the world of venture capital, and particularly detailed at Facebook.

Facebook comes out of the book particularly poorly, as if Zuck is a some kind of fickle boy king holding court with his sycophants. During his time at Facebook, the author appears to try to join this clique, only to grow bitter when entry is rebuffed.

Most interesting is the description of Facebook's struggle with advertising revenue, especially after its IPO. As the author describes it, Facebook couldn't figure out how to make the promised revenue. Eventually, in mid-2013 or so, they found a way, not by using data on what people do, but knowing who most people are, which turned out to be particularly important on mobile ("basic targeting like age and gender was a godsend to data-starved marketers ... data-wise, you have a first-party relationship with [only] a few apps"). The real value of Facebook turned out not to be its data on what people are doing, but merely being able to identify most people consistently and willing to exploit that to its fullest.

It helps if you know at least a few of the personalities featured in the book. Paul Graham, Sam Altman, Chris Sacca, Greg Badros, and many others make at least brief appearances, usually to get splattered with the slime that drips from these pages. Many VCs and people at Facebook and Twitter are also mentioned, mostly described as the amoral who's who of the rich and powerful of Silicon Valley.

Like many who got lucky, the author confuses luck with skill. Sure, that pitch meeting went well, but that meeting almost didn't happen. Success often was a result of a chance connection at the right time. In cases where the author angered someone with his arrogance or foolishness, someone should have killed the deal, and might have had they been in a slightly different mood that day. This startup was almost stillborn, barely making it into Y-combinator. The acqui-hire almost didn't happen, almost killed by lack of customer growth and shenanigans by the author. That everything worked out even as well as it did was mostly good fortune.

To his credit, the author realizes some of this in the end. In the acknowledgments, he writes, "Let's be blunt: ours was a relationship of pure convenience, and I exploited you as much as you did me." But he also writes of some he encountered, "In a Valley world awash with mammoth greed and opportunism masquerading as beneficent innovation, you were the only real loyalty and idealism I ever encountered." I'd like to think mammoth greed and opportunism have much smaller representation than idealistic innovation.

Some may call me wishful, but I think pushing for that idealistic world to be true is part of making it true. This book is not going to stop me from thinking that tech companies should be a force for idealistic innovation and promise for the future. At least in my circles, most people I talk with are awash with idealism, a genuine belief that what they are working on can make things better for others. It saddens me to see that the author's perception of the tech industry is so different than my own.

Friday, November 25, 2016

Quick links

Some of the tech news I found interesting lately, and you might too. Heavy on the comics this time to lighten the mood:
  • Jeff Bezos: "Good leaders ... seek to disconfirm their most profoundly-held convictions, which is very unnatural for humans ... Anybody who doesn’t change their mind a lot is dramatically underestimating the complexity of the world we live in." ([1])

  • Amazon is hiring 120k employees just for the holidays. I can't believe how our baby is all grown up. ([1])

  • On building products: "Keep it extremely simple, or two thirds of the population can’t use your design" ([1] [2])

  • "The problem isn't the users: it's that we've designed our computer systems' security so badly that we demand the user do all of these counterintuitive things." ([1])

  • Fun AI experiments from Google. Don't miss "Quick, Draw!" ([1])

  • Interesting new phone design, screen taking up the entire front: "Hands down, the best looking smartphone ever" ([1])

  • Great article on Netflix recommendations, tidbits on the importance of reacting immediately to new data, using immediate intent, freshness (esp. new releases), and perceived quality (difference between online evaluation and offline). ([1])

  • Opinionated summary of RecSys 2016, and also somewhat of a summary of recommendations and personalization research as of 2016 ([1] [2])

  • Xavier Amatriain on lessons learned from building recommender systems ([1])

  • YouTube is now using deep learning for recommendations, more than just embeddings, includes a ranker with heavily engineered features ([1])

  • Ex-Facebook employee: "News Feed optimizes for engagement. As we've learned in this election, bullshit is highly engaging." ([1])

  • Pfeffer: "You need to be careful with what you measure, because you are going to get it, and often you don’t really want it." ([1] [2])

  • Obama: "Traditionally, when we think about security and protecting ourselves, we think in terms of armor or walls. Increasingly, I find myself looking to medicine and thinking about viruses, antibodies." ([1])

  • Surprising, just set up a hotspot, and the interference from people's fingers moving in the WiFi signal is enough to catch most of the passwords anyone enters while connected ([1] [2])

  • "An entire company’s product line has just been turned into a botnet that is now attacking the United States" ([1] [2])

  • short URLs hid malicious content that was then used to get at Colin Powell's e-mail ([1])

  • Carefully picked textures on eyeglass frames to fool face recognition, pictures in the paper are amusing ([1] [2])

  • AI guru Andrew Ng: "We're lucky the AI community is very open, and top researchers freely share many ideas and even code. This helps the whole field progress. Hope we can keep it that way." ([1] [2])

  • Love this: "Being able to go from idea to result with the least possible delay is key to doing good research" ([1])

  • Two new massive labeled open data sets from Google, one for images, one for videos ([1] [2])

  • "Translations that are vastly improved compared to the previous phrase-based production system. GNMT reduces translation errors by more than 55%-85% on several major language pairs" ([1])

  • Google CEO Sundar Pichai: "Our goal is build a personal Google for each and every user." ([1])

  • I got a mention in The Guardian for some of my past work: "Greg Linden may not be a household name..." ([1])

  • Data on what Amazon Echo is actually used for. Mostly playing a song, it appears. ([1])

  • Like at the last dot-com boom, there are a bunch of delivery services cropping up with models that don't seem like they're likely to be profitable. Uber, which was in a better position than most to do this profitably, just shut their food delivery service down, which doesn't bode well for the others. ([1])

  • Current state of virtual reality: "None of these uses are particularly compelling right now, especially given the cost of buying a VR headset. This may change in the future." ([1])

  • "Giving employees hours, days or even months in which to work without close scrutiny has enhanced productivity instead of harming it" ([1])

  • T-mobile's CEO on leadership: "Listen to your employees, listen to your customers, shut the f*** up, and do what they tell you" ([1])

  • SMBC comic on survivorship bias ([1])

  • SMBC comic on work, dark but funny: "The important thing is to find the low low bar that works for YOU." ([1])

  • SMBC comic on eliminating security risks ([1])

  • SMBC comic being a scientist. Don't miss the mouseover: "Hopefully your kids don't drink as much as research scientists" ([1])

  • Love the mouseover text on this SMBC comic: "Studying social science has completely obliterated my ability to enjoy pleasant human behaviors" ([1])

  • SMBC comic on political economy. Don't miss the mouseover text on humans: "In economics, they're robots. In political economy, they're all jerks. In sociology, they're all misunderstood." ([1])

  • Brilliant comic by Matthew Inman (The Oatmeal) on happiness and meaning ([1])

  • Xkcd on a CS degree: "That just means I understand how everything went so wrong" ([1])

Sunday, August 28, 2016

More quick links

A tightly curated list of what has caught my attention lately:
  • New Yorker on AI: "A lot of what people are calling 'artificial intelligence' is really data analytics -- in other words, business as usual. If the hype leaves you asking 'What is A.I., really?,' don’t worry, you're not alone .... Intelligent software helps us interact and deal with the ... [information] onslaught ... winnowing an increasing number of inputs and options in a way that humans can’t manage without a helping hand .... A set of technologies that try to imitate or augment human intelligence .... [But] we are a long way from creating virtual human beings ... In the meantime, we're going to have to deal with the hyperbole surrounding A.I." ([1])

  • Tim O'Reilly: "Humans are increasingly going to be interacting with devices that are able to listen to us and talk back .... [Alexa] demonstrates that conversational interfaces can work, if they are designed right .... Smaller domains where you can deliver satisfying results, and within those domains, spend a lot of time thinking through the 'fit and finish' so that interfaces are intuitive, interactions are complete, and that what most people try to do 'just works'." ([1])

  • Netflix: "We think the combined effect of personalization and recommendations save us more than $1B per year" ([1] [2] [3])

  • "The main reasons cited for using ad blockers include avoiding disruptive ads (69%), ads that slow down their browsing experience (58%) and security / malware risks (56%). Privacy wasn’t the top answer. So Facebook thinks if its can make its ads non-interruptive, fast, [useful,] and secure, people won’t mind." ([1] [2])

  • According to the NYT, Uber lost $1.2B on $2.1B in revenue in H1 2016 ([1] [2])

  • "Amazon reaches new high of 268,900 employees — skyrocketing 47% in just one year" ([1])

  • Amazon's going hard for Netflix on their key vulnerability, strength of the catalog ([1])

  • Great example of how Bezos sees failure as just a step toward success, following up on their $170M loss from an expensive Amazon Fire Phone with another (and I think very promising) attempt using existing cheap phones ([1] [2])

  • Talks from ScaledML 2016, including Jeff Dean, Qi Lu, Ilya Sutskever, and more ([1] [2])

  • Great paper on the data pipelines at Facebook and some of their design tradeoffs ([1])

  • Good article on Facebook's approach to research, not separate from engineering, not part of engineering, but just open ([1] [2])

  • Great article in ACM Queue on Amazon's microservices, which allows for "permissionless innovation" and has many benefits for testing, deployment, debugging, and reliability ([1] [2])

  • Nice example of fine-grained control of data center power and cooling using machine learning to save electricity ([1])

  • Precision agriculture using GPS, self-driving tractors, and crop and nutrient sensors ([1])

  • Pew Internet study of Amazon Mechanical Turk (MTurk), lots of remarkable details, including that most workers are making less than $5/hour, almost all less than $8/hour ([1])

  • "The line between outright deception and poor user design is often hard to distinguish" ([1])

  • "[The] many confusing design decisions made us wonder if projects were assembled entirely from poor stackoverflow posts" ([1] [2])

  • Amusing story of what happens when a geolocation is missing ([1])

  • On education: "A feeling of hopefulness actually leads us to try harder and persist longer -- but only if it is paired with practical plans for achieving our goals, and specific, concrete actions we’ll take when and if (usually when) our original plans don’t work out as expected." ([1])

  • On management: "We have to give them the space to fail in the short term so they can succeed and grow in the long term ... There is that magical moment when we delegate and allow an emerging leader to grow into their new responsibilities, and they end up being way better at it than we ever were. That’s real management success." ([1] [2])

  • On teams: "The best teams respect one another’s emotions and are mindful that all members should contribute to the conversation equally ... A shared belief that it is safe to take risks and share a range of ideas without the fear of being humiliated." ([1] [2])

  • Comic on being data-driven and how it sometimes feels ([1])

  • Xkcd on self-driving cars: "This car has 240% of a horse's decision-making ability" ([1])

  • Xkcd: "Is this a normal bug?" ([1])

  • Xkcd on code quality ([1])

  • SMBC comic on statisticians ([1])

  • SMBC comic on economists and the golden goose, don't miss the mouseover text: "A physicist would figure out how the Goose was transmuting elements without getting to a high temperature, then use the trillions of dollars to build a really sweet fleet of quadcopters" ([1])

  • SMBC comic that perfectly captures why I love talking with geeks, it's the infectious enthusiasm ([1])

Thursday, June 02, 2016

Quick links

A tightly curated list of what I enjoyed in the news recently:
  • Bezos: "Every single important thing we’ve done has taken a lot of risk, risk-taking, perseverance, guts, and some have worked out. Most of them have not." ([1])

  • Bezos: "You need to select people who tend to be dissatisfied ... As they go about their daily experiences, they notice that little things are broken in the world and they want to fix them. Inventors have a divine discontent." ([1])

  • Page: "Is it going to affect everyone in the world? Very few ... think this way." ([1])

  • "More than anything else, the rise of the bots signals the death of the mobile app ... The whole app thing didn't really work out." ([1] [2])

  • "As it turns out, the mundanity of our regular lives is the most captivating thing we could share with one another" ([1])

  • "This is the most demonically clever computer security attack I've seen in years ... insert a nearly undetectable backdoor into the chips themselves" ([1])

  • "Most Android vulnerabilities don't get patched. It's not Google's fault. It releases the patches, but the phone carriers don't push them down to their smartphone users ... This is a long-existing market failure." ([1])

  • "It’s not like iPhones have somehow gotten worse. Other phones, though? They’ve gotten a whole lot better. And they’re cheap." ([1])

  • "Google, with its tech chops and its control over digital ad delivery, is positioned to do what individual publishers and their associations can’t do on their own, though, by requiring that ads are not obtrusive or annoying — a main reason people choose to block ads." ([1])

  • "How quickly cars can learn to do the really hard parts of driving ... navigate congested cities in the pouring rain where humans, pets and rodents run into the road" ([1] [2] [3])

  • "With so many advances in machine learning recently, it’s not unreasonable to ask: why aren’t my recommendations perfect by now?" ([1])

  • "Developers’ speed mattered ... only to the extent that we made effective product design choices ... It didn’t matter how fast they were moving if they were moving in the wrong direction." ([1])

  • "Building and growing startups may appear glamorous from the outside ... It is anything but that from the inside." ([1])

  • "% of pitches for bots and/or AI companies approaching 100%" ([1])

  • "Tech firms are plundering departments of robotics and machine learning ... for the highest-flying faculty and students, luring them with big salaries ... The field was largely ignored and underfunded during the 'AI winter' of the 1980s and 1990s, when fashionable approaches to AI failed to match their early promise." ([1])

  • The FizzBuzz Tensorflow interview "will probably only make sense to people who have gone through really terrible CS interview processes" ([1] [2])

  • Remarkable, deep networks trained on artistic style, then used to apply those styles to video ([1])

  • A good summary of the state-of-the-art in deep learning ([1])

  • "There are limits to the predictive abilities of even tremendously superior intelligence (due to partial observability, chaotic behavior, or sheer randomness)" ([1] [2])

  • SMBC comic: "Once you realize there is no hope, you can relax and just enjoy the progress in machine learning." ([1])

  • My favorite old T-shirt from, Earth's Biggest Bookstore ([1])

Saturday, May 14, 2016

Code Monster from Crunchzilla is now open source

Code Monster from Crunchzilla is now open source, free to use and modify.

Code Monster is a tutorial that has been used by hundreds of thousands of children around the world to learn a little about programming. It's a series of short lessons where each lesson involves reading and modifying a small amount of code. Changes to the code show up instantly, students learning by example and by doing.

The lessons content for Code Monster from Crunchzilla is in a JSON file that can be modified fairly easily to create your own content. By open sourcing Code Monster from Crunchzilla, I hope three things might happen:
  1. Translations. Taking the current content and translating into languages other than English for use in more classrooms around the world.

  2. New lessons and new content. By adding new messages and example code to the JSON lessons file, new tutorials could be created for teaching programming games, working through puzzles or math problems, or perhaps a more traditional computer science curriculum aligned with a particular lesson plan.

  3. Entirely new tutorials. Some ideas and techniques used by Code Monster, such as how Code Monster provides informative error messages, how it does live code, or how it avoids infinite loops in students' code, might be useful for others creating web-based coding environments.
Code Monster from Crunchzilla has been used in computer labs and classrooms around the world. One of the most common requests is translations into languages other than English. Now that the code is open source, I hope that makes it easier for translated and modified versions to get in front of even more children.

If you use the code for anything that helps children learn computer programming, I'd love to hear about it (please post a comment here or e-mail me at

Saturday, April 02, 2016

Quick links

What has caught my attention lately:
  • "We simply don't know how to securely engineer anything but the simplest of systems" ([1])

  • Impressive at their scale: "Facebook ... releases software ... three times a day" and makes configuration changes "thousands of times a day... every single engineer can make live configuration changes." ([1]) 

  • Pew Research report on global internet and smartphone usage ([1])

  • Cute idea for telepresence: "We propose projecting [2D] virtual copies of people directly onto (potentially irregular) surfaces in the physical environment" ([1])

  • For those of us tracking virtual reality, a detailed review of the Oculus Rift ([1]), a review of Hololens ([2]), and a fun TED talk motivating augmented and virtual reality ([3])

  • For disk to be the new tape "custom disk designs uniquely targeting cold storage" are required that are "much larger, slower, more power efficient and less expensive." ([1]) Related, Google seeks new disk designs ([2])

  • Lessons from building AWS, including automate everything and favor primitives over frameworks ([1])

  • In the AWS service terms: "However, this restriction will not apply ... [when] human corpses to reanimate and seek to consume living human flesh, blood, brain or nerve tissue." ([1])

  • Google says, "With multi-homing ... failover, recovery, and dealing with inconsistency ... are solved by the infrastructure, so the application developer gets high availability and consistency for free and can focus instead on building their application" ([1] [2])

  • Remarkably successful contest: "The winning team exceeded the power density goal for the competition by a factor of 3 ... Some of us at Google didn’t think such audacious goals could be achieved." ([1])

  • "Welcome to the Internet of Things... and its tradeoffs" ([1] [2] [3])

  • Netflix's catalog has dropped to 5,532 titles from 8,103 titles in about two years ([1] [2])

  • "The James Webb Space Telescope will be a major advance ... primary mirror will be 50 times [larger] ... eight times the resolution" ([1])

  • "The price of planetary insurance, it turns out, isn’t all that high." ([1] [2])

  • Teaching math: "In most people’s everyday lives ... what [people] do need is to be comfortable reading graphs and charts and adept at calculating simple figures in their heads ... Decimals and ratios are now as crucial as nouns and verbs." ([1])

  • He's the "‘seagull of science.’ He used to fly in, squawk, crap over everything, and fly away." ([1])

  • Good answer to the question, "What are the most important things for building an effective engineering team?" ([1]) Related, similar advice from Amit Singh ([2] [3])

  • An old office map from early 1997 (back when Amazon only sold books, "Earth's Biggest Bookstore"). My "office" was a card table in a kitchen. ([1])

  • What If comic: What would happen if you tried to squeeze all the water going over Niagara Falls into a straw? It's worse than you'd think. ([1])

  • Xkcd comic on bots: ""Python flag: Enable three laws" ([1])

  • Good Xkcd comic on Celsius or Fahrenheit ([1])

  • SMBC comic: "Philosophy tip: Make any sentence profound by adding 'true' to it" ([1])

  • Dilbert comic: "No need for conversation. I know everything about you." ([1])

  • Comic with a Calvin and Hobbes crossover into Bloom County, brings back memories ([1])