Overview
- Irregular’s testing of ChatGPT, Claude, and Gemini found repeated structures and limited character choices, indicating outputs that only look random.
- In one Claude sample, 50 prompts produced just 23 unique 16‑character strings, with a single password repeated 10 times and no repeated characters across outputs.
- Estimated entropy for LLM‑generated 16‑character passwords was about 20–27 bits versus roughly 98–120 bits for truly random strings, implying far faster brute‑force cracking.
- Common password‑strength meters rated these strings as strong, but researchers say the tools miss model‑specific patterns that dramatically reduce security.
- Searches on GitHub revealed LLM‑style password fragments in real projects, and follow‑up tests saw Gemini 3 Pro display a built‑in warning and recommend passphrases and password managers.