by
new York
CNN
—
As the tech sector rushes to develop and deploy a slew of powerful new AI chatbots, their widespread adoption has sparked a new set of data privacy concerns among some companies, regulators and industry watchers.
Some companies, including JPMorgan Chase (JPM), have cracked down on employee use of ChatGPT, the viral AI chatbot that started Big Tech’s AI arms race, due to compliance issues related to employee use of third-party software.
This only heightened growing privacy concerns when OpenAI, the company behind ChatGPT, revealed it had to temporarily take the tool offline on March 20 to fix a bug that allowed certain users to see other users’ chat history subject lines.
The same bug, now fixed, also allowed “some users to see first and last name, email address, payment address, last four digits (only) of a credit card number and another active user’s credit card expiration date,” OpenAI said in a blog post.
And just last week, Italian regulators issued a temporary ban on ChatGPT in the country, citing privacy concerns after OpenAI disclosed the breach.
“The privacy considerations with something like ChatGPT cannot be overstated,” Mark McCreary, co-chair of the privacy and data security practice at law firm Fox Rothschild LLP, told CNN. “It’s like a black box.”
With ChatGPT, which launched to the public in late November, users can generate essays, stories, and song lyrics just by typing prompts.
Since then, Google and Microsoft have also rolled out artificial intelligence tools, which work in the same way and are powered by large language models trained on vast amounts of online data.
When users enter information into these tools, McCreary said, “You don’t know how it’s going to be used next.” This raises particularly strong concerns for businesses. As more employees casually embrace these tools for work email or meeting notes, McCreary said, “I think the ability for company trade secrets to be deposited in these different AIs will just increase.”
Steve Mills, chief AI ethics officer at the Boston Consulting Group, also told CNN that the biggest privacy concern most companies have around these tools is the “inadvertent disclosure of sensitive information.” .
“You have all these employees doing things that might seem very innocuous, like, ‘Oh, I can use this to summarize notes from a meeting,’” Mills said. “But by pasting the meeting notes into the prompt, you’re suddenly, potentially, leaking a whole bunch of sensitive information.”
If data entered by people is used to further train these AI tools, as many companies behind the tools have stated, then you have “lost control of that data, and someone else has it.” “, added Mills.
OpenAI, the Microsoft-backed company behind ChatGPT, states in its privacy policy that it collects all kinds of personal information from people who use its services. It indicates that it may use this information to improve or analyze its services, conduct research, communicate with users and develop new programs and services, among others.
The privacy policy states that it may provide personal information to third parties without notice to the user, except as required by law. If the 2,000+ word privacy policy seems a bit opaque, that’s probably because it’s pretty much become industry standard in the internet age. OpenAI also has a separate terms of service document, which obliges the user to take appropriate measures when using its tools.
OpenAI also released a new blog post on Wednesday outlining its approach to AI security. “We don’t use data to sell our services, advertise, or create profiles of people — we use data to make our models more useful to people,” the blog states. “ChatGPT, for example, is improving with additional training on the conversations people have with it.”
Google’s privacy policy, which includes its Bard tool, is also lengthy and offers additional terms of service for its generative AI users. The company states that to help improve Bard while protecting user privacy, “we select a subset of conversations and use automated tools to help remove personally identifiable information.”
“These sample conversations may be reviewed by trained reviewers and retained for up to 3 years, separately from your Google Account,” the company says in a separate FAQ for Bard. The company also cautions, “Do not include information that can be used to identify you or others in your conversations with Bard.” The FAQ also states that Bard conversations are not used for advertising purposes, and “we will clearly communicate any changes to this approach in the future.”
Google also told CNN that users can “easily choose to use Bard without saving their conversations to their Google Account.” Bard users can also view their prompts or delete Bard conversations via this link. “We’ve also implemented safeguards designed to prevent Bard from including personally identifiable information in his responses,” Google said.
“We’re still learning exactly how this all works,” Mills told CNN. “You just don’t know exactly how the information you put in, if it’s used to retrain those patterns, how it manifests as outputs at any given time, or if it does.”
Mills added that sometimes users and developers don’t even realize the privacy risks lurking with new technologies until it’s too late. One example he cited was the early auto-complete features, some of which ended up having unintended consequences, such as entering a social security number that a user started typing – often at the alarm and surprise the user.
Ultimately, Mills said, “My current view is that you shouldn’t put anything in these tools that you don’t want to assume will be shared with others.”
