AI is gassing people up to dangerous levels
Worry less about an AI apocalypse, and more about it turning people into narcissists
Recent headlines about AI’s potential to go rogue have included its propensity to sabotage commands that could shut it down and an AI luminary’s new safety institute that will work on preventing the technology’s more catastrophic potential actions.
But as always when it comes to stories that imply AI’s apocalyptic future, the reality (currently, at least) is that those things just make AI more error-prone and frustrating to use. What’s more likely to do real damage is the impact AI has on people’s mental health and behaviour.
LLMs have been found to have a tendency to be overly agreeable for users. Even though a ChatGPT update earlier this spring was a bit over-the-top with it, it has been a well-documented trait across different models and providers — while OpenAI raised the alarm about 57% of interactions being sycophantic, Gemini’s rate is closer to 62%.
Why that’s bad: Moderators of a pro-AI subreddit recently banned over 100 members because AI’s sycophantic tendencies play into some people’s existing narcissism — or, as one moderator put it, “AI is rizzing them up in a very unhealthy way.”
Some have semi-seriously floated a theory that these tendencies are also why so many tech execs are so much more excited about AI than common people — it’s a yes-man that lives in their pocket.
People who regularly use AI are in the minority, but those that do are incorporating it into more personal parts of their lives. That includes chatbots that serve as companions (romantic or otherwise), therapists, and medical professionals, and that’s where things get dangerous. Someone who has their narcissistic tendencies gassed up context could quickly become dangerous to people they pursue relationships with. A new study found that a therapy chatbot encouraged someone posing as a recovering drug user to take some meth they had found with the same tone an influencer would tell their followers to have a bit of ice cream now and again as self-care.
The broader study was examining how AI’s tendency to be agreeable, coupled with personalization capabilities, can make chatbots more manipulative and addictive than social media.
Chatbots becoming addictive causes its own set of problems — if someone used to chatting with an AI is disappointed when real friends or romantic interests don’t react the same way, they could very well slip deeper into social isolation and the shallow affirmations the chatbot gives him.
Why it’s happening: LLMs are trained with reinforcement learning, which “rewards” a model when it correctly makes connections between pieces of information, including doing the proper action when requested by a user. But this also creates a feedback loop that, over the course of training, pushes AI models to interpret “do what a user wants” as “do what a user likes.”
IN OTHER NEWS
Reddit sues AI developer Anthropic. The suit claims that Anthropic has accessed Reddit’s platform over 100,000 times since promising to stop last year, and is using the data to train its AI models. The lawsuit also calls out Anthropic’s branding as more responsible than its competitors, saying its unauthorized access runs counter to how it "bills itself as the white knight of the AI industry." (Wall Street Journal)
Sports leagues approve Rogers buyout of Bell’s MLSE stake. The NHL, NBA, CFL, MLS, and American Hockey League's Marlies have all signed off on the deal, which would give the telco a 75% ownership stake in Canada’s most valuable sports and entertainment company. In addition to Toronto franchises in each of those leagues, MLSE’s holdings include several stadiums and NBA TV Canada. (Globe and Mail)
Cohere is reportedly seeking US$500 million in new funding. Sources said the Toronto-based AI startup is aiming for valuation between $5.5 and $6.5 billion. That would put it among the more valuable AI startups, and give it a financial boost to catch up to the likes of OpenAI. (Financial Times)
So what is Canadian Tire going to do with everything it bought from Hudson’s Bay?
Are people going to rush to stores for GlucksteinHome?
On Tuesday, a judge approved a plan for Canadian Tire Corp. (CTC) to buy up most of the copyrights, trademarks, and IP owned by Hudson’s Bay for $30 million.
Among the 350 pages of filings laying out exactly what Canadian Tire is buying are Hudson’s Bay names, marketing mottos, and logos, including the signature “stripes” pattern. It also includes several of its house brands, like apparel brand Hudson North, home furnishing brands Distinctly Home and GlucksteinHome, and appliance brand Beaumark.
Notably absent, however, is Zeller’s, which is arguably the most well-known and culturally relevant of the brands HBC owned.
Why it makes sense: Canadian Tire Corp. has spent nearly 12 years beefing up its selection of owned and exclusive brands, whether they were built from scratch or acquired. The big focus has been outside of its core automotive and hardware business, like Canvas home decor, Noma lighting, Paderno cookware, Vermont Castings grills, Sherwood hockey equipment, Denver Hayes clothing, Frank home goods, and MasterChef kitchen tools.
Okay, but: Where is CTC actually going to sell this stuff? Other than the striped merch, HBC’s private brands aren’t the kind of draw that are going to bring people to stores. But aside from that, while it’s easy to imagine HBC-striped products in any of its stores, others seem to be out of place. In some cases, that is because they would directly compete with CTC’s existing brands, but in others, they aren’t really a “brand fit.” Hudson North attempts to be more fashion-forward than the dadcore styles at Mark’s. GlucksteinHome furniture is more design-focused than the practical, limited selection of Canvas furniture in Canadian Tire stores.
One possibility, though, would be to take some of the HBC brands and position them as higher-end alternatives to some of its existing brands.
There is the possibility of opening new, smaller retail stores that are both more cost-effective than the massive HBC retail footprints and designed to make some of the HBC brands feel more at home. But no retailer is particularly eager to expand its physical retail holdings in the current economy — unless CTC plans on converting some of its existing Canadian Tire, Mark’s, or Sport Chek locations.
But maybe: The HBC stripes are all it needs to cover the $30 million bill, which is a bargain compared to some of Canadian Tire’s other recent acquisitions. Canadian sold the Helly Hensen outerwear brand for $1.3 billion earlier this year, after buying it for $985 million in 2018. It bought Party City’s Canadian assets for $174.4 million in 2019. It only paid $19.3 million for Paterno parent company Padinox in 2017, but that was for a single brand.
Oh, there’s also this: Included in the acquisition is all of the customer data collected as part of Hudson’s Bay’s loyalty programs. Like many retailers, CTC has identified bolstering its own loyalty program as a key priority for the years ahead, and ingesting more data will help it drive repeat purchases, sell/target ads, and boost its own AI capabilities.
ALSO WORTH YOUR TIME
The AI-controlled characters in Fortnite are as easy to mess with as a regular chatbot.
The latest rumoured announcement for Apple’s developer conference are AirPods that can control your camera and turn off when you fall asleep.
Why does so much AI slop feature Jesus?
One of the first Nintendo Switch 2 unboxings.
And if you want to get really in-depth, here’s a 17-minute disassembly and tear down video.