Large Language Model (LLM)-based agent systems are emerging as a new software paradigm and have been widely adopted across diverse domains. As agent systems are inevitably prone to bugs and continually evolve to meet changing external requirements, maintaining them is therefore critical yet requires substantial effort. In this talk, we will present our recent work on maintaining and optimizing agent systems. First, we will discuss common quality issues that arise during agent maintenance and explore how existing software maintenance techniques perform when applied to agent systems. Second, we will address the cost challenges of multi-agent systems and introduce our recent budget-aware optimization technique for multi-agent systems.