2024 - Year of Generative AI at Work

Implementing Generative AI Solutions at scale

This substack draws inspiration from by the paper “Generative AI at work” by Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond (2023). This research is so far my favorite study in the social sciences that examines empirically what generative AI can actually do for businesses.  They studied how a conversational assistant transform the work of more than 5000 customer support agents (think about call representatives at call centers, or agents that chat live with customers). The authors found that using the tool increases productivity by 14% on average, measured by number of issues resolved per hour. More importantly, they found that this tool affects novice workers more than experienced workers, 34% improvement for beginners while doesn’t affect highly skilled workers. They argue that Generative AI disseminates the best practices of more able workers, and assists newbies. Furthermore, they found that AI assistants improve customer sentiments, increase employee retention, and may lead to worker on-the-job learning. They then suggest that access to generative AI can increase productivity, with more gains on the inexperienced workers. 

Image generated by Dalle using prompt: “Generate an image for an AI summit. Remove people, only keep a logo.”

The research setup takes place at a Fortune 500 tech company, and the customer support agents are those that perform live chats with customers. This setup is a typical human in the loop experient where agents can ignore the chatbot’s suggestions, and proceed based on their own experience, knowledge if they choose to do so. This allows for the agents to exercise their own autonomy over their work, advice, and the quality of service that they provide to customers. Regarding the quality of work, the researchers found that off-shore customer service reps are less tired if they work overnight shifts to serve US-based customers. Customers escalated to managers less often. This leads to lower worker attrition, which often caused by newer workers leave. Overall, this research suggests positive changes to both employee satisfaction, and customer satisfaction after the AI-assistant was introduced to customer-service work. This optimistic outlook on the potential of generative AI on workers’ productivity and business gains propelled me to start this substack.

This past week, I attended the AI summit in New York City. The conference gathers technologists, journalists, business people, a lot of established companies as well as startups that are working on different AI applications. One could clearly see generative AI to be the main theme at the conference. Meta announced their Purple LLama at the summit. A Google representative talked about Gemini. OpenAi talked about the one year anniversary of ChatGPT. Everyone else was busy trying to make generative AI work at different types of organizations from small & medium businesses to large corporations. Someone even said during a panel that a year ago we never heard of RAG, retrieval-augmented generation; now it’s basically the industry-standard of how to implement an information retrieval system to exploit all the good promises of large language models. What I gathered from these different conversations at the Summit was that the year of 2024 will be the year when everyone tries to make generative AI work. 

The concept of generative AI at work is front and center, and has become more important than ever. Many open questions have opened up for technologists, AI ethicists, legal and compliance officers, and social scientists to answer. For technologists, the main question is how to move from toy demos to scalable solutions that could be sustained by themselves, how do we avoid the pilot purgatory. For system designers, and AI solutions engineers, the question is how to combine different methods to reliably engineer solutions for any problem at hand. We now can combine LLMs with traditional software engineering. How can we creatively, cheaply and quickly  deploy our solutions, to bring our solutions to market, thus gain more from GenAI? For compliance officers, the questions are how can we avoid all the pitfalls of generative AI such as hallucinations, copyrighting, etc? For AI ethicists, what does it mean to create company policy around the use of generative AI such that we develop, deploy and use them responsibly given financial and time constraints? For social scientists, what does it mean by having a culture of generative AI at work? What does this culture look like? Whom does it privilege? Who will lose out at work? 

All of these questions require a lot of brain power and human resources to figure out. As for me personally working at the intersection of both technological development, business development, and AI ethics, I often ask the question: how I can develop, and deploy a solution that is beyond toy solution, while staying  engaged with AI ethics folx, and social scientists to think about what the downstream effects on different groups of end users might be. 

All of these are the motivating factors for the inception of this substack. I hope that this will be an intellectually exciting space for me to explore different practical as well as theoretical questions when implementing generative AI solutions in the work place not only for increased productivity, improved customer experience, but also for increased work satisfaction, and a more just society.