ChatGPT‘s Hidden Army: The Untold Story of AI‘s Contract Workers

In just a few short years, artificial intelligence (AI) has become a ubiquitous presence in our daily lives. From Siri and Alexa fielding our queries to customer service bots handling our complaints, AI-powered systems have rapidly become the interfaces through which we interact with the digital world. Tech giants like OpenAI have made headlines with impressive breakthroughs like ChatGPT, a large language model capable of engaging in fluid conversation and generating human-like text on demand.

But behind the dazzling algorithms and sleek interfaces lies a hidden economy of human labor responsible for making the AI magic possible. To function, AI systems require vast quantities of training data that has been labeled, categorized and fed to machine learning models by an army of invisible workers. These AI trainers spend their days performing repetitive microtasks – labeling images, moderating content, transcribing audio, writing snippets of text – for a few cents per task. It‘s tedious, often low-paid work, but it plays an absolutely vital role in enabling AI systems to understand and interact with the messy human world.

The People Behind the AI Curtain

Alexej Savreux knows this world intimately. The 34-year-old from Kansas City spends his days working as an AI trainer, helping to provide the data that powers OpenAI‘s widely-used ChatGPT language model. His tasks include labeling photos, subjectively rating the quality of AI-generated text, and dreaming up prompts and responses to train the model to engage in more natural conversation.

"We are the ones doing the grunt work that makes the magic happen," says Savreux. "You can have the fanciest neural networks in the world, but it‘s all just a black box without good training data. That‘s where we come in."

Savreux is just one of a vast pool of workers, likely numbering in the hundreds of thousands globally, fueling the ongoing AI boom with their data work. The types of tasks performed span a wide range:

• Image labeling: Identifying and labeling objects, text, faces, etc. in images/video
• Data categorization: Sorting data like text, images, products into specified categories
• Content moderation: Reviewing and flagging problematic content like hate speech
• Audio transcription: Converting recorded speech to text
• Linguistic annotation: Identifying parts of speech, named entities, sentiment in text
• Relevance scoring: Rating how relevant a result is for a given query
• Chatbot training: Writing prompts and responses to train conversational AI
• Text generation: Writing, completing or editing text based on prompts
• AI output evaluation: Rating quality or accuracy of AI-generated text/images
• Autonomous vehicle annotation: Labeling pedestrians, vehicles, signs, lanes in footage

Labeling each individual piece of data can take anywhere from a few seconds for a simple image categorization to 5-10 minutes for a complex 3D sensor fusion annotation. Multiply that by the billions of data points needed to train modern AI systems across thousands of domains and you start to get a sense of the immense scale of the human knowledge work involved. By some estimates, a single AI system can require up to 1.1 million taskshours of human annotation behind the scenes.

"What I always say is, if you‘re amazed at an AI system, you should be even more amazed at the human effort that went into building it," notes Mary Gray, a senior principal researcher at Microsoft Research and co-author of Ghost Work, a book examining the lives of workers powering the digital economy behind the scenes. "The paradox of automation is that the desire to eliminate human labor always generates new tasks for humans."

The Piecemeal World of Crowdwork

So who exactly are the people carrying out this work? And under what conditions? For all their importance in enabling the AI resurgence, data labelers are often invisible and undervalued, classified as independent contractors and limited to the piecemeal wages and precarity of gig work.

Much of the data annotation work is parceled out on digital crowdsourcing platforms like Amazon Mechanical Turk (MTurk), Appen, and Playment. Initially conceived as a way to outsource microtasks that were difficult for computers but easy for humans, these platforms serve as marketplaces to connect "requesters" (often tech companies) posting tasks with workers around the world looking to complete them for pay.

The basic model works like this: a requester uploads a batch of data needing annotation along with instructions and a price per task. Workers browse available tasks and claim them on a first-come basis, completing them and submitting for review and payment, then moving on to the next microtask. Requesters can set qualification criteria for tasks (location, language, quality rating, etc.) and freely reject work they deem unsatisfactory with no pay.

For workers, access to these gigs is relatively open compared to traditional freelance tech jobs – all you generally need is an internet connection, computer, and to be over 18. But the work is decidedly precarious. On MTurk, the biggest platform, workers are classified as independent contractors rather than employees, meaning they are not guaranteed minimum wage, overtime, or benefits. Rejection of work is common with no formal appeals process. And competition for tasks is fierce with workers across the globe racing to grab work as soon as it‘s posted, leading to long unpaid waiting times between tasks.

Pay rates for individual tasks are paltry – typically ranging from a few cents to a dollar or two, with more complex tasks like 3D sensor fusion for autonomous vehicles on the higher end. A 2018 study by the United Nations‘ International Labour Organization examining pay conditions on major crowdwork platforms found that the average worker spends 20 minutes of every hour on unpaid work (searching for tasks, reading instructions, submitting work, waiting for review, etc.) and earns just $2.16/hr on average after accounting for unpaid time.

Studies have repeatedly found that significant proportions of workers on MTurk and other major platforms rely on the gigs as a primary source of income, despite the low and unreliable wages. A Pew Research survey in 2016 found that 25% of workers on MTurk said the money earned there was "essential" for meeting their basic needs, while another 37% said it was "important." Notably, much of the workforce demographics skews heavily international, with workers concentrated in India and other countries in the Global South.

Labeling a Troubling Web

Pay isn‘t the only concern when it comes to data labeling. Content moderation – reviewing and flagging disturbing and graphic content to help AI systems detect it automatically – makes up a significant portion of the work and can take a serious psychological toll.

"Reviewing content on the internet for 8 hours a day, you‘re exposed to conspiracy theories, hate speech, violence," explains Sarah Roberts, an assistant professor at UCLA who has studied commercial content moderation for over a decade. "There‘s research showing that there is a real psychological impact of repeated exposure to that type of disturbing content."

Social media giants have come under fire in recent years after reports exposed harrowing working conditions and lack of support for their outsourced moderators, many based in the Philippines. Workers reported reviewing hundreds of violent, racist and sexually explicit posts per day, leading to symptoms of PTSD, anxiety and sleep disorders.

While the biggest platforms like Facebook, Twitter and YouTube have since moved to bring more moderation in-house and provide dedicated counseling and support, much of the labeling work farmed out for AI training datasets receives far less scrutiny and resources. Stories abound on online forums from data annotators exposed to disturbing content with no mental health support or even warning of what they‘d be reviewing.

Some companies have aimed to partially address the issue by chunking and decontextualizing the content into bits and pieces. The disturbing image is sliced into small sections which are sent out for labeling with no indication to the worker of the whole. But it‘s a bandaid on a gaping wound as the work of cleaning up the web‘s underbelly continues to be foisted onto the most vulnerable.

The Boom Behind the Boom

The hidden human workforce behind AI is likely to only grow in the coming years. As AI is integrated into more and more domains and products, the need for niche, application-specific training data is expected to explode. The data annotation tools market alone is projected to grow to over $7 billion by 2028 at a CAGR of over 30%.

"I anticipate that we‘ll see specialization," says Dr. Katy Khamash, a crowdwork researcher at Oxford University. "So instead of just labeling ‘car‘ or ‘pedestrian‘, it will be labeling Ferraris vs. Fords or Gucci handbags vs. generic. There will be increased demand for more situational, domain-specific data collection and labeling for each specific business use case."

Some of the biggest tech giants have already moved to meet the surging demand by acquiring data labeling platforms outright. Apple acquired Seattle startup Mighty AI in 2019 to accelerate training data production for its autonomous driving project. A year later, Uber snatched up Mighty AI competitor Spare5. Google, IBM and Salesforce have all made significant investments into data labeling platforms to build out their own proprietary AI training data pipelines.

"Building out internal pipelines enables a significant competitive advantage when it comes to iterate rapidly on model development," notes Judith Donath, author of The Social Machine, a book examining human-AI interaction. "It‘s a recognition that agile access to highly tailored training data is becoming a key differentiator in the AI race."

The rapid growth of the "AIaaS" (AI as a Service) sector is spurring demand further as dozens of startups spring up to sell tailored AI solutions for every conceivable business use case. Off-the-shelf AI products for everything from content recommendation to demand forecasting offer the promise of AI to non-tech companies – and most rely heavily on outsourced data work behind the scenes.

"For a lot of businesses that don‘t have dedicated machine learning teams, the real barrier is not the modeling – it‘s getting the right training data," explains Donath. "So we‘re seeing a lot more specialty data labeling providers emerge for specific verticals like medical imaging, retail visual search, industrial defect detection, etc."

The Future of Ghost Work

While data labeling work is booming now, its future is far from certain. Growing automation threatens to put many out of a job, as AI systems are trained to handle more data annotation tasks recursively. A 2021 study by OpenAI found that machine learning models like CLIP had already progressed to a level where they could effectively replace human labelers for certain image classification tasks. Earlier this year, Stability AI released an open source machine learning model that can generate images from text descriptions, significantly reducing the need for manual image labeling.

"It‘s always been clear that data labeling was an intermediate job that will likely disappear," notes Donath. "As the models get better, they‘ll require less and less human hand-labeling to function."

Some experts predict that within 5-10 years, most data labeling work done by humans today will be automated away. That could spell disaster for a ghost workforce that has grown increasingly dependent on the gigs to make ends meet. With few worker protections and a globalized labor pool, collective organizing and advocacy will be an uphill battle.

"The decentralized, remote nature of the work makes it incredibly challenging to organize for better conditions," notes Mary Gray. "There have been efforts to create guilds and unions for crowdworkers, but progress has been limited so far."

Still, some see potential for data labeling work to provide a stepping stone into the tech economy if working conditions and pay are improved. Labeling data for AI provides exposure to core concepts of machine learning and Python is often necessary to complete tasks efficiently. With the proper training and career pathways, motivated workers could conceivably upskill into higher-paying machine learning engineering and data science roles over time.

"In the best case scenario, this could become a bridge into the knowledge economy for talented workers in the developing world," notes Khamash. "But it will require a fundamental shift in how we structure and value this work. Companies need to see investing in their data labeling workforce as investing in the long-term sustainability of their AI supply chains."

In the meantime, efforts are underway by researchers and activists to push for more visibility and protections for ghost workers. Initiatives like FairWork and Turkopticon let workers rate conditions across platforms so they can make more informed decisions about which gigs to take. Others are pushing for policy changes like mandating minimum wage for crowdwork and requiring companies to provide more transparency to users about their AI supply chains. Groups like the Gig Worker‘s Collective are experimenting with grassroots organizing and collective bargaining to win better conditions platform by platform.

"At the end of the day, all of us are increasingly relying on the insights and capabilities enabled by this human-labeled data," says Gray. "We need to start asking ourselves, do we want to live in a world where we treat the humans in the loop as expendable and invisible? Or do we want to recognize their vital role in the AI we‘re building and fight to ensure they can do that work with dignity?"

For Savreux, it‘s a matter of basic respect for the work he and his fellow labelers do every day to make services like ChatGPT possible.

"I take a lot of pride in what I do because I can see the amazing progress happening in AI and know that my work is a core part of that," he says. "But pride doesn‘t pay the bills. We deserve to be recognized, to earn a living wage, to have a seat at the table in defining the future we‘re helping to build."

As the revolution in generative AI accelerates, the struggle for equity and representation for ghost workers toiling behind the scenes is only just beginning. The road ahead is long and strewn with obstacles. But if the history of labor has taught us anything, it‘s that collective power is always the key to bending the arc of innovation toward justice. The ghost workers may be invisible now, but they won‘t stay silent forever.