Back to Blogs

AI at Crossroads: Navigating AI Safety in the Evolving Startup Landscape

Navigating AI Safety in the Evolving Startup Landscape

Published on Dec 04, 2023

Following a series of dramatic events, OpenAI's internal challenges, and more importantly, AI safety, have come into the spotlight. These events have uncovered potential gaps in the market, offering an opening for emerging startups to innovate and differentiate themselves.   

Concerns for AI Safety  

While OpenAI transitioned into a for-profit model in 2019, its parent company and major shareholder remains the non-profit OpenAI Inc., and the board members are responsible for upholding their public charter. The charter, released in 2018, commits to providing society with appropriate tools to navigate the path to artificial intelligence. However, it is also cognizant of the potential dangers of the technology.   

The charter reads, "...we expect that safety and security concerns will help in reducing the traditional publishing in the future as well as assist in increasing the importance of sharing safety, policy, and standards research."  

The members of the board individually have also been vocal about their concerns. Ilya Sutskever, co-founder and the Chief Scientist at OpenAI, harbors deep concerns about the dangers of artificial intelligence. In July 2023, Sutskever created a Super Alignment team inside the company that would explore ways to ensure that their technology was safe and could not be used for harmful activities. Helen Toner expressed her concerns regarding the commercialization of their technology. In a paper published in October this year, Toner criticized OpenAI for releasing ChatGPT at the end of last year and sparking "a sense of urgency inside major tech companies" and prompting competitors to "accelerate or circumvent internal safety and ethics review processes" to ensure they did not fall behind.  

Read more: The Evolving Business Landscape and Future of Generative AI.  

Startup Landscape

OpenAI's rapid commercialization of technology outpacing developments in safety standards has been the central theme of recent events. Since its inception in 2015, the company has raised $11.31 billion, with a majority of investments coming in from Microsoft. In addition to its flagship large language model ChatGPT, OpenAI released DALL-E, an AI system adept at creating original artwork from text descriptions, a marketplace to create personalized AI applications to expand its consumer base, and is currently exploring opportunities to make its own AI chips. Additionally, the company unveiled a data partnership program through which it would create an open-source dataset for training language models to make responses more human-like. These developments have been in just the past year with only slight headway in the advancements to develop guardrails that foster responsible deployment.    

In such a climate, wary about the potential of AI, Anthropic, a California-based artificial intelligence company, seems to be treading the correct path. The creators of Claude 2, a foundation model competing against ChatGPT, are focused on machine learning safety and reliability. Co-founded by two seasoned industry professionals and former research executives from OpenAI, Anthropic's safety concerns prompted it to delay the release of the first version of Claude developed in 2022. This month, the company announced its partnership with its investor, Google, to advance AI safety standards and responsible development. Anthropic has raised a total of $7.25 billion in funding at a valuation of $25 billion with investments from Google, Amazon, and Zoom Ventures, among others.  

Read more: Investment Outlook: Private Equity Market Trends 2023     

Future Landscape for AI Startups  

While safety concerns have been at the forefront, the administrative vulnerabilities unveiled within OpenAI have brought up questions regarding the company's future, which are yet to be clarified. Additionally, the saga has created concern among customers about whether they should rely on only one kind of large language model for their AI strategies. This has made space in the artificial intelligence ecosystem for startups to step in.   

AI Safety  

Anthropic is a major contender to leverage the shake-up at OpenAI. Amid the drama, it released the latest version of Claude, which outperforms existing chatbots by its capacity to handle larger token prompts. This translates to more contextual and efficient responses, setting it apart in the market. Hugging Face has been making long-term plans for its technology deployment instead of fast-paced commercialization. The French-American company has raised a total of $400 million at a $4.5 billion valuation from an array of investors, including IBM, Google, Qualcomm, Nvidia, and Sequoia Capital. Cohere, a Canadian company, provides AI technologies specific to their customer's needs. The company's CEO reported they have seen an increase in traffic and inquiries about their technology since the drama unfolded. Cohere has raised a total of $440 million at a $2.95 billion valuation from investors, including Nvidia, Salesforce Venture, and SAP SE, among others.  

Read more: From Recovery to Reliability: Technology's Impact on Supply Chains  

The generative AI industry is poised to grow at a CAGR of 42%, from $40 billion in 2022 to $1.3 trillion in the next ten years. In the past year, OpenAI has established itself as the dominant player within the industry with a thriving product in ChatGPT, backing from Microsoft, and multiple anticipated projects underway. However, the events of the past few weeks have highlighted crucial concerns that impact the broader landscape. Its fast-paced commercialization outpacing advancements in the standards for safety have reignited the conversation about the safe deployment of AI technologies. Additionally, unveiled weaknesses within the firm have created opportunities for alternative startups to push for dominance in this trillion-dollar industry.  

SG Analytics, recognized by the Financial Times as one of APAC's fastest-growing firms, is a prominent insights and analytics company specializing in data-centric research and contextual analytics. Operating globally across the US, UK, Poland, Switzerland, and India, we expertly guide data from inception to transform it into invaluable insights using our knowledge-driven ecosystem, results-focused solutions, and advanced technology platform. Our distinguished clientele, including Fortune 500 giants, attests to our mastery of harnessing data with purpose and merging content and context to overcome business challenges. With our Brand Promise of "Life's Possible," we consistently deliver enduring value, ensuring the utmost client delight.      

Partner of choice for lower middle market-focused Investment Banks and Private Equity firms, SG Analytics provides offshore analysts to support across the deal life cycle. Our complimentary access to a full back-office research ecosystem (database access, graphics team, sector & and domain experts, and technology-driven automation of tactical processes) positions our clients to win more deal mandates and execute these deals in the most efficient manner. 


Contributors