Large language models (LLMs) are rapidly transforming the practice of software engineering. Code correctness has been the primary focus for evaluating these models, but real-world software development requires much more. In this talk, I will discuss how we could move beyond correctness to address broader dimensions of LLM-generated code. I will use my recent work on efficiency, fairness, and diversity as examples, and discuss the challenges and opportunities that lie ahead in advancing LLMs for code.