For all their impressive capabilities, large language models (LLMs) often fall short when given challenging new tasks that require complex reasoning skills.