John Draper could exploit it with his Captain Crunch whistle

Security expert Bruce Schneier has pointed out that large language models are open to the same kind of weaknesses that phones in the 1970s had, the kind that John Draper exploited.

Does not do much else than stress you out

While ChatGPT might have the "brains" to pass a medical exam, a recent study published in the journal JAMA Oncology shows that the AI provides unhelpful information on cancer treatment, potentially harming patients who are already stressed out and looking for answers.

Gets data from search results 

The people who bought you the Brave browser have improved its search engine to use a large language model to produce answers to your search queries.