Twitter Shops have Content Moderation Problem

Shopping segments on Twitter pose content moderation risks. They could be used “in a way that leads to an individual or societal harm,” according to an internal memo sent to a crew of Twitter employees.

Twitter shopping components could lead to ‘individual or societal harm,’ according to a leaked memo.

The memo was sent in early July to a bunch of Twitter employees across different teams, according to a person with details of it. In the email, some living and not-yet-released features of Twitter shopping are classified as high risk, with the email warning that content temperance hasn’t been prioritized in shopping.

Twitter shopping permits brands to list items for sale and pin a handful of products at the top of a merchant’s profile. However, unlike similar features on Instagram, users can’t buy the product directly on Twitter — items for sale instead link to a merchant’s website. An extended version of the shop module was submitted earlier this year, allowing sellers to list and showcase up to 50 outcomes in their storefront, and it was available to all US merchants as of June.

In an excerpt of the memo titled “risk assessment,” several aspects of Twitter’s e-commerce tool are categorized as “high.” For example, one high-risk problem is merchant-generated fields like shop terms and descriptions, which bad actors could use the memo alerts in dangerous ways.

Twitter’s shop feature permits anyone with a professional account trading items in the US to manually add products for sale to their profile. In addition, when selecting items in the expanded Twitter shop, merchants can count a custom shop name and report directly on the Twitter dashboard. It’s in these fields that people working in shopping say there are risks.

The memo smudges the picture of a bare-bones process to look for and remove potentially abusive or harmful content on Twitter shopping. According to the notice, the platform doesn’t yet have a policy on what is considered a shop name or description violation and lacks guidance on handling breaches. The memo also states that Twitter lacks the tools to witness violations in shop names and reports and that there is no way for users to register storefronts for content in these fields.

The main selling point of Twitter’s shopping features is shareability. The company has presented other updates like reminders that permit customers to get notified and tweet about new releases from brands. Twitter users can now click into merchant shops, view products, and click out to merchant websites, but sharing the storefronts isn’t feasible.

The internal memo lists the power to share storefronts as “high” on its risk assessment, stating that if the attribute were released, it could further boost harmful content, increasing the visibility of content that disregards Twitter rules.

According to the memo, Twitter has some automated detection mechanisms for individual products listed for sale. But proactive steps to detect violations are “limited,” and the company has the least staff and tools for further review.

“Shareable Shops, therefore, improve the likelihood that users may notice violative Shops or violative goods included in a Shop,” the memo reads. “It may also incentivize bad actors to amplify destructive or violative goods by tweeting and sharing their Shop.”

 Twitter spokesperson Lauren Alexander confirmed the memo’s authenticity and said it was part of a new feature assessment led by the product trust team. Alexander says the assessments are designed so different groups can provide input to ensure safe new product releases.

“We’re always working to enhance the safety of our service, and that is particularly true for adding new products and features,” Alexander says. “We have intentionally taken the time to test our new shopping surfaces before scaling or expanding to new markets. As a result, we can gather learnings to better inform our approach to health and safety and develop additional infrastructure and policy needs.”

Twitter has been attempting to diversify its revenue sources beyond advertising recently, experimenting with a paid subscription service, ticketed live broadcasts on Spaces, and paid Super Follows. Though the platform has been rolling out and expanding its commerce tools, it’s lagged behind other social networks like Instagram and TikTok; in his attempted takeover of Twitter, Elon Musk indicated that payments had the potential to be a substantial chunk of the company’s business.

Twitter’s former head of security, Peter “Mudge” Zatko, said the company suffered from negligent security and bot removal issues, among other problems, in a wide-ranging whistleblower report. Shortly after news of the complaint, The platform would combine a “health” team working on reducing misinformation and other toxic content with a services team that works on spam accounts.

Leaks and details about such a feature have been circulating for months, including Tuesday’s announcement that initially indicated that Twitter is working on the quality. In May, app investigator Jane Manchun Wong shared screenshots of a segment called Twitter Notes in some places and Twitter Articles in others.

It allows users to write formatted blog posts with links, pictures, and embedded tweets. Nima Owji, an app researcher, shared screenshots of the same tool in April, which allowed users to share posts with their followers or complete standalone connections for commands to communicate elsewhere on the web.

Adding a long-form script to Twitter could drastically change the platform’s character, which a short-hand form has long defined. Twitter is arguably already full of comprehensive written screeds, shared in the form of threads of tweets or screenshots of others’ writings or users’ writing.