Why can't ChatGPT recite lyrics perfectly, or tell you the contents of books it knows? Let's discuss what LLMs are bad at...and why.