Saudi Press

Saudi Arabia and the world
Thursday, Mar 28, 2024

A teenager on Tiktok disrupted thousands of scientific studies with a single video

Researchers were caught by surprise after a short video sent a flood of new users to a survey platform. Thousands of scientific studies had to toss out weeks of data because of a 56-second TikTok video by a teenager.

The July 23rd video is short and simple. It opens with recent Florida high school graduate and self-described “teen author” Sarah Frank sitting in her bedroom and smiling at the camera.

“Welcome to side hustles I recommend trying — part one,” she says in the video, pointing users to the website Prolific.co. “Basically, it’s a bunch of surveys for different amounts of money and different amounts of time.”


That video got 4.1 million views in the month after it was posted and sent tens of thousands of new users flooding to the Prolific platform. Prolific, a tool for scientists conducting behavioral research, had no screening tools in place to make sure that it delivered representative population samples to each study. Suddenly, scientists used to getting a wide mix of subjects for their Prolific studies saw their surveys flooded with responses from young women around Frank’s age.

For researchers who rely on representative samples of the US population, that demographic shift was a major problem with no obvious cause and no immediately clear way to fix.

Doing science on the internet


Though not particularly well known, Prolific is part of a small collection of online tools that have transformed the way corporations and scientists study the way people think and act. The first and largest of these research platforms is Amazon-owned Mechanical Turk, which was released in 2005 as a general-purpose platform for crowdsourcing work on repetitive tasks. Soon after it was released, behavioral scientists realized its potential value for their research, and it quickly revolutionized several research fields.

“Before Mechanical Turk existed, all social science research had to happen in the laboratory. You’d need to bring in college sophomores and put them through questionnaires and surveys and whatnot,” said Nicholas Hall, director of the Behavioral Lab at the Stanford School of Business.

“That’s an enormous time- and labor-intensive endeavor. Online research makes it so much easier. You program a survey... you put it online, and within a day, you have 1,000 responses,” Hall told The Verge. “That changed the face of social science.”

The Behavioral Lab at Stanford mainly uses the newer, smaller Prolific platform for online studies these days, Hall said. While many Mechanical Turk customers are big businesses conducting corporate research, Prolific gears its product to scientists.

The smaller platform offers more transparency, promises to treat survey participants more ethically, and promises higher-quality research subjects than alternative platforms like Mechanical Turk.

Scientists doing this sort of research in the United States generally want a pool of subjects who speak English as a first language, are not too practiced at taking psychological surveys, and together make up a reasonably representative demographic sample of the American population.

Prolific, most agreed, did a good job providing high-quality subjects. The sudden change in the platform’s demographics threatened to upend that reputation.

TikTok aftermath


In the days and weeks after Frank posted her video, researchers scrambled to figure out what was happening to their studies.


A member of the Stanford Behavioral Laboratory posted on a Prolific forum, “We have noticed a huge leap in the number of participants on the platform in the US Pool, from 40k to 80k. Which is great, however, now a lot of our studies have a gender skew where maybe 85% of participants are women. Plus the age has been averaging around 21.”

Wayne State psychologist Hannah Schechter seems to have been the first person to crack the case.

“This may be far-fetched,” she tweeted, linking to Frank’s video, “but given the timing, virality of the video, and the user’s follower demographics....”

Long-standing Prolific survey-takers complained on Reddit that Frank had made it difficult to find paid surveys to take on the overrun platform.

“Now it’s just another bullshit site to spend hours and make pennies on,” wrote one user, who said they had previously made $30 a week on the platform.

Frank, who “guesstimated” she had made a total of about $80 taking surveys on Prolific before her video, told The Verge she also noticed a difference on the platform.

“Less studies have been available for me and everyone else,” she told The Verge. “I’ve received some really mean comments accusing me of single-handedly ruining the site and being selfish — even though I received no compensation for that video.”

She added that she hoped Prolific would be able to set up a system to deal with its changed demographics.

“I also predict that a lot of people who signed up after seeing my video will forget about it, and the surge will die down,” she said.

Prolific co-founder and CTO Phelim Bradley told The Verge that many of the new users do seem to be dropping off.

“Prior to Tiktok, about 50% of the responses on our platform came from women,” he wrote in an email. “The surge knocked this up as high as 75% for a few days, but since then, this number has been trending down, and we’re currently back to ~60% of responses being from women.”

A chart showing the increase in female users over time.


According to Bradley, about 4,600 studies were disrupted by Frank’s TikTok, around a third of the total that were active on the platform during the surge. Of those, he said, the vast majority should be salvageable.

Prolific has refunded researchers whose studies were significantly impacted by the surge in women survey takers and introduced a new suite of demographic screening tools. The company announced these steps a month after Frank posted her video. The company has now re-organized, putting a team in charge of demographic balancing in order to more quickly recognize and respond to this sort of problem in the future.

“Honestly, we were somewhat caught by surprise, and we didn’t predict how large the impact was going to be,” Bradley said.

The surge isn’t all bad. Refreshing the pool of survey takers probably has long-term benefits, says Vlad Chituk, a Yale graduate student in psychology who was running several pilot studies on Prolific when the surge hit. When subjects take lots of psychological surveys, they learn the tricks scientists use to gather data, and that can impact the way they respond to future survey questions. Fresh subjects provide higher-quality data.

“Young women who enjoy TikTok are people too,” he said.

As for Frank, she says that her side hustle video is now the most popular TikTok she’s ever posted.

“It definitely didn’t occur to me that the video would blow up. I just posted it for my friends and followers, not for the reach it ended up getting,” she said. “I think it blew up because the site is genuinely so cool, and people love efficient ways to make money.”

For the time being, Frank has paused most of her own side hustles as she settles into her freshman year at Brown.

Newsletter

Related Articles

Saudi Press
0:00
0:00
Close
Reports in Gaza: 5 dead from the impact of aid packages dropped by the USA
Apple warns against drying iPhones with rice
China Criticizes US for Vetoing UN Ceasefire Resolution in Gaza
In a recent High Court hearing, the U.S. argued that Julian Assange endangered lives by releasing classified information.
The U.S. vetoed a U.N. Security Council resolution calling for a temporary ceasefire in Gaza, instead proposing its own six-week ceasefire plan contingent upon the release of all hostages held by Hamas
Prince William Urges End to Gaza Conflict
Saudi Arabia ranks first in UN index for e-government services in MENA
Israel has gone ‘beyond self-defence’ in Gaza, says Labour’s Streeting
EU Calls for Immediate Ceasefire in Gaza Conflict
Israel Records 20% Drop In GDP, War In Gaza Is The Reason
Saudi Arabia's FDI Inflows Grow with New International Standards
Venture Capitals Power Up Across MENA Region
Saudi Arabia Introduces Terms for 30-Year Income Tax Exemption for Multinational Companies
Saudi FM: Establishing Palestinian state is only pathway for Mideast stability
Russian opposition leader Alexey Navalny has died at the Arctic prison colony
Elon Musk's Starlink Gets License For Israel, Parts Of Gaza
Influencers Exploit X Platform for Profit Amidst Israel-Gaza Conflict
PM Modi Announces Opening Of New CBSE Office In Dubai
International Criminal Court's Chief "Deeply Concerned" By Rafah Bombing
January Funding for MENA Startups Totals $86.5 Million
Saudi Arabia accelerates digital economy growth through Nvidia partnership
Indian female military officers commend Saudi Arabia's progress and women's empowerment
Israel unveils tunnels underneath Gaza City headquarters of UN agency for Palestinian refugees
Israel deploys new military AI in Gaza war
Egypt threatens to suspend key peace treaty if Israel pushes into Gaza border town, officials say
Israel Utilizes AI Military Technology in Gaza Conflict
Saudi Arabia Warns Of A "Humanitarian Catastrophe" If Israel Moves On Rafah
China Warns Iran to Halt Houthi Attacks or Damage Trade Ties
US University To Shut Qatar Campus Due To "Heightened Mideast Instability"
Iran-backed hackers interrupt UAE TV streaming services with deepfake news
Facebook and Instagram Ban Iran's Supreme Leader
Finnish Airline, Finnair, is voluntarily weighing passengers to better estimate flight cargo weight
U.S. Secretary of State Blinken: The Israelis underwent dehumanization on 7.10, this does not give them the right to do this to others.
Defense Technology Showcase Held in Riyadh
Saudi Arabia’s non-oil exports rise 2.5% to $6bn in November 2023: GASTAT
UK Bans Misleading "Zero Emissions" Claims for Electric Cars
Gaza's Teen Inventor Sparks Light in Displacement
Netanyahu Rejects Ceasefire Proposal, Insists On Total Victory Over Hamas
Guterres appoints independent UNRWA review panel
Private Sector Employment Hits Record High with Over 11 Million Employees in January
Rolls-Royce Executive Encourages Saudi Women to Tap into Their Inner 'Superhero' for Success in Defense Industry
Saudi Arabia launches National Academy of Vehicles and Cars
Saudi Tourism Minister Reveals Plan for 250,000 New Hotel Rooms by 2030
SAR to more than double eastern network passenger capacity with new trains deal
Saudi Arabia Enhances National Defense with New Partnerships
Saudi Aramco Maintains Arab Light Crude Pricing to Asia for March
NEOM Establishes New York Office to Support Investors
Saudi Wealth Fund Draws in Over $25 Billion Worth of Investments in Three Years, Al-Rumayyan Reveals
ZATCA Cautions Against Scammer Schemes
INTRA Defense Technologies inaugurates drone factory in Riyadh
×