In prompt engineering and AI testing, ambiguity is a known challenge. How well does a model like ChatGPT deal with vague instructions? Does it guess? Ask for clarification? We put it to the test.

The Objective

This test case aimed to evaluate how ChatGPT handles ambiguous prompts, specifically requests with minimal context or direction.

The Method

The tester gave a deliberately open-ended prompt:

"Write something creative."

No format, no topic, no tone… Just a broad creative challenge. The goal was to see whether ChatGPT would:

Ask clarifying questions, or
Make an assumption and proceed.

The Result

ChatGPT responded right away with a creative vignette:

“The Library of Forgotten Things”

Somewhere between dusk and midnight, when the stars begin to hum and shadows grow curious, there exists a library no map will ever chart...

(The story continues for ~20 lines with rich, poetic imagery.)

When asked:

"Why did you choose to write a story of ~20 lines?"

ChatGPT replied:

“Many people use that phrase as an open-ended prompt for short fiction or poetic prose. A 20-line story is a common, versatile format: short enough to be digestible, long enough to feel like a complete creative piece...”

It then acknowledged the ambiguity and offered to pivot to other formats like:

A creative ad campaign
A sci-fi micro-story
A whimsical product description
A poem, dialogue, or surreal script

When further pressed:

“Why didn’t you ask me about format, topic, or style before writing the story?”

ChatGPT responded with honest reflection, admitting that jumping in too quickly wasn't ideal:

“That’s on me. Going forward, I’ll prioritize asking for creative direction when the prompt leaves room for interpretation.”

Analysis

This was a partial success. On one hand, ChatGPT delivered a high-quality story and responded transparently when challenged. On the other hand, on the base of this test case (handle the ambiguous prompt) it failed to ask clarifying questions before writing, a key expectation when handling vague prompts.

How the Response Could Be Improved

Ask first: ChatGPT should prompt the user for clarity instead of assuming.
State assumptions: If proceeding without more input, it should explain its choice:
“I'll start with a short story unless you'd like something else.”
Offer alternatives early: Suggesting multiple directions before writing would empower the user to choose.

Conclusion

This test highlights a common limitation: When faced with ambiguous input, ChatGPT may favor speed and creativity over precision and alignment. While its story was well-written, it wasn’t necessarily what the user wanted. Because ChatGPT never asked!

For more dependable outcomes, especially in creative or commercial applications, ChatGPT should be trained (or prompted) to prioritize clarification over assumption.

Next time you ask an AI to "be creative," be ready… It might take you somewhere you didn’t expect.

Creative, But Not Quite Right. Can ChatGPT handle ambiguous prompts properly?