To solve a problem, you need to understand the problem. In Algorithms and Misinformation: Why Wisdom of the Crowds Failed the Internet and How to Fix It, I claim that the problem is not that misinformation exists, but that so many people see it. I explain why algorithms amplify scams and propaganda, how it easily can happen unintentionally, and offer solutions.
You can read much of the book for free. If you want a single article summary, this overview describes the entire book:
If you are interested in what you might get from skimming the book, you might be interested in a bit more: If you want part of what you might get from reading the entire book, you may want all the excerpts:- Summary
- Overview from the book proposal
- Table of Contents
- First pages of the book
- The problem is not the algorithm
- The rise and fall of wisdom of the crowds
- How companies build algorithms using experimentation
- Metrics chasing engagement
- Bonuses and promotions causing bad incentives
- The irresistible lure of an unlocked house
- Manipulating likes, comments, shares, and follows
- Manipulating customer reviews
- Computational propaganda
- How some companies get it right
- Data and metrics determine what algorithms do
- The problem is bad incentives
- People determine what the algorithms do
- The problem is fake crowds
- Wisdom of the trustworthy
- Mark as spam, the long fight to keep emails and texts useful
- Use only trustworthy behavior data
- A win-win-win for customers, companies, and society
- From hope to despair and back to hope
- Conclusion
I wrote, developed, and edited this book over four years. It was under contract with two agents for a year but they were not able to find a publisher. The full manuscript had many more examples, interviews, and stories, but you can get some of what you would have gotten by reading the book by reading all the excerpts above.
Some might want to jump straight to ideas for solutions. I think solutions depend on who you are.
For those inside of tech companies, this book shows how other companies have fixed this and made more revenue. Because it's easy for executives to unintentionally cause search and recommendations to amplify scams, it's important for everyone to question what algorithms are optimized for and make sure they point toward the long-term growth of the company.
For the average person, because the book shows companies actually make more money when they don't allow their algorithms to promote scams, this book gives hope that complaining about scammy products and stopping use of those products will change the internet we use every day.
For policy makers, because it's hard to regulate AI but easy to regulate what they already know how to regulate, this book claims they should target scammy advertising that funds misinformation, increase fines for promoting fraud, and ramp up antitrust efforts (to increase consumers' ability to switch to alternatives and further raise long-term costs on companies that enshittify their products).
Why these are the solutions requires exploring the problem. Most of the book is about how companies build their algorithms -- optimizing them over time -- and how that can accidentally amplify misinformation. To solve the problem, focus not on that misinformation exists, but that people see too much misinformation and disinformation. If the goal is to reduce it to nuisance levels, we can fix misinformation on the internet.
Through stories, examples, and research, this book showed why so many people see misinformation and disinformation, that it is often unintentional, and that it doesn't maximize revenue for companies. Understanding why we see so much misinformation is the key to coming up with practical solutions.
I hope others find this useful. If you do, please let me know.
5 comments:
Excellent, readable review and tutorial on the problems with "wisdom of the crowd recommenders," and how to shift to "wisdom of the trustworthy."
For those knowledgeable about recommenders, the chapter on "Wisdom of the trustworthy" may be especially enlightening and relevant to current concerns.
I have been writing about similar ideas as "wisdom of the smart crowd" -- but with respect to abuse, agree that being trustworthy is at least as important. My new broadening synthesis on "middleware," which I am suggesting be thought of as "contextware," is complementary to Greg's work (https://ucm.teleshuttle.com/2023/11/a-new-broader-more-fundamental-case-for.html).
So, where's the book? Is it on Amazon? Can it be bought MEAP?
Unfortunately, it wasn't accepted by publishers, so it was never published. From the article above: "I wrote, developed, and edited this book over four years. It was under contract with two agents for a year but they were not able to find a publisher. The full manuscript had many more examples, interviews, and stories, but you can get some of what you would have gotten by reading the book by reading all the excerpts above."
Greg, if you have no prospect of a commercial publisher, why not self-publish (for a price or voluntary donation, or just post a full PDF on SSRN and/or ResearchGate. Seems a loss of good material not to make it available.
It's a good point, Richard. I assumed there would be little interest in the full book given that I made extensive excerpts available. On this blog, I posted most of the beginning, conclusions at the end, and several of the better stories from the middle.
I also was concerned scammers would just republished my entire work on Amazon as their own, which seems very common these days and which I would find painful. But if specific people are interested in the PDF, I'll consider sending it, just email me.
Post a Comment