Randy Newman’s ingenious lyricism earned him devoted fans but not mainstream success. His turn to Hollywood would bring wider recognition.