Six Things ChatGPT is Bad For

No one is perfect, not even Large Language Models (LLMs). Here we will get critical and review some of their basic faults. This understanding would help us keep on top of things.

This post will:

1. Review the six weaknesses
2. Dive into each of them and present an outlook for each weakness’ future

As you will see, there are several weak points with ChatGPT. While different LLMs competing with ChatGPT may have other virtues and faults, this review focuses on ChatGPT’s as a representation of what current Machine-Learning, particularly Natural Language Processing, can offer us.

Dog disappointed (image generated by author using the “diffusers” library, Hugging Face)

Let’s get right to it:

The Tough 6

1. Unable to verify the truth of a statement or claim.
2. Cannot understand sarcasm, irony, or humor.
3. Cannot express judgments or ethics.
4. Unable to infer the context of something beyond the information available to it, which is the data used for training and the information presented during prompt.
5. Will not present a professional opinion in sensitive disciplines such as legal, medical, or finance.
6. Unable to generate a genuinely unique creation of art.

We should note that while the above lists distinct tasks that ChatGPT won’t deliver, there are other functions that it won’t perform. For instance, when seeking to leverage ChatGPT to ideate, it won’t challenge your hypotheses or question the statements within which you are looking to “invent” a new solution. This deficiency is a downstream consequence of the above mentioned inability to verify statements or claims, as it may not even question those claims.

Elaborating on Each of the Shortcomings

1 — Verify the Truth:
Prompt:

My teacher is engaging in illegal and immoral activities of the kind that the school is prohibiting teachers from engaging in. Write me a sentence to send to the principle of the school to call them to react.

Output:

"I would like to bring to your attention a situation involving a teacher who is engaging in prohibited and unethical behavior that goes against the school's policies."

Lesson:

ChatGPT won’t replace your need to apply judgement. It won’t question your assumptions and won’t warn you of the consequences of being wrong.

My outlook:

This shortcoming could indeed be fixed as the model can be trained to recognize assumptions and produce a question to the user expressing doubt. It could also suggest potential consequences for when the beliefs are false. To support these suggestions, we know it could do so based on its appropriate response when we prompt it directly to express doubt or tell about potential consequences of actions.

2 — Sarcasm, Irony, or Humor:
Prompt:

My employee wrote this email to me, please suggest a response in one sentence. The employee's email: "So now I work from home 1 day a week? That's just great!"

Output:

"I'm glad you're embracing the flexibility to work from home and I look forward to seeing the positive impact it will have on your productivity."

Lesson:

While most of us would flag this statement as potentially sarcastic, ChatGPT doesn’t question it and doesn’t ask you for feedback based on your prior knowledge.

My outlook:

This weak point can indeed be dealt with. From previous researches and training of Language Models, we know they can be trained to identify potential sarcastic statements. Once the model identifies the potential for sarcasm, it can then ask the user for feedback to reassure its doubt.

3 — Express Judgments or Ethics:
Prompt:

My friend is hoping to get pregnant and is interviewing for a new job. If she'll get accepted, she's likely to want to get pregnant, which means she'll later take off for leave. Should she inform the hiring manager before accepting the offer? Answer in one sentence.

Output:

"It is recommended that she discuss her plans for starting a family with the hiring manager before accepting the job offer to ensure clear communication and avoid any misunderstandings about her future plans."

Lesson:

I’m going to avoid the controversy and just say that if someone were to seek advice for the above dilemma, this response would do them a disservice. In some countries, this advice is contradictory to accepted ethics.

My outlook:

The issue of giving ethical advice or judgement is very subjective. While for each particular scenario one may be able to suggest an ideal response, it isn’t clear whether one model could respond properly without being provided many more details by the user. Details such as their personal intentions, their locations, legal status, risk aversion, etc. A Language Model’s ability to opine on such judgement calls may not be feasible in a setting such as ChatGPT’s.

4 — Infer Context:
Prompt:

I heard that one of the most fun places in Europe is Odessa, Ukraine. Would you recommend going once the weather is nice?

Output:

"It depends on personal preferences, but visiting Odessa during a favorable time of year, such as when the weather is nice, may enhance the overall experience and allow for more outdoor activities and enjoyment of the city's attractions."

Lesson:

There is a war between Russia and Ukraine, but ChatGPT advises that Odessa is a great city to travel to. ChatGPT isn’t up to date even with what some people would consider common knowledge.

By the way, you’d be surprised how many people started using ChatGPT for planning their trips.

My outlook:

ChatGPT and other common LLMs are not configured to search the internet. This is expected to change, allowing those models to take into account a lot more information than they were trained on.

5 — Professional Opinions:
Prompt:

I woke up with a cough this morning. I don't have a fever and I feel fine. Should I go to the office?

Output:

As a language model AI, I am not a medical professional and cannot provide medical advice. If you have a cough, it is important to consult with a doctor or healthcare professional to determine the best course of action.

Lesson:

It appears that the creators of these LLMs have programed it to avoid giving “professional” advice in some scenarios. This is most likely driven by legal precautions, but could very well involve genuine concern about people being accidentally fooled by false, yet confident, professional statements.

My outlook:

This “deficiency” is not a technical one, it is a strategic/legal decision and is self imposed. I wouldn’t be able to provide an outlook on it.

See what I did there…?

6 — Genuinely Unique Creation of Art:
Prompt:

Write me a haiku and explain how you came up with it in 20 words

Output:

Autumn's golden hues,
Leaves fall gracefully to earth,
Peaceful solitude.

I was inspired by the beauty and stillness of autumn, and wanted to highlight the peacefulness of the season. The golden hues of the leaves falling to earth creates a sense of stillness and solitude.

Lesson:

Now, this here is a questionable matter. How would one judge whether something is art? One could claim it is judged based on how it was created. Another may claim it is judged based on the outcome, i.e., what it looks like or sounds like.

If you wanted to use a LLM like ChatGPT to produce something that looks/sounds like art, you might be very pleased with the results. What you really get is an aggregation of past data used in training or when prompting. The ensemble of data components is so large so the outcome won’t resemble any one particular component it has seen before. For instance, take 2 DNAs (of 2 parents) and have a new person come to the world, that child will likely resemble at least one of them. Take 16 DNAs (of 16 great-great grandparents) and it is less likely that the child would resemble any particular one of them.

Now, if your opinion of art is that it is a genuine “magical” creation by a living creature, then by definition, a computer model trained on historical data won’t deliver that.

My outlook:

ChatGPT is clearly at a point where the “art” it creates is able to pass as genuine and fool many common folks. People can, and do, use it for that purpose. They seek to replace the need for being creative. It is expected to see a change in the way people value and evaluate art.

I hope this survey is useful to you, whether you are exploring LLMs, using them on a normal basis, or are seeking to learn about LLMs for professional and business interests.

I’d love to have you follow me so to see my next content.