Recently, the Software Engineering and Intelligent Computing research team led by Professor Li Zheng from the School of Information Science published a regular paper titled "Prompt Alchemy: Automatic Prompt Refinement for Enhancing Code Generation" in IEEE Transactions on Software Engineering.
This work focuses on the performance of Large Language Models (LLMs) in code generation tasks and proposes a new framework called Prochemy for automatically optimizing and improving LLM prompt in code generation tasks. Prochemy first constructed a training data set for evaluation, which includes existing data and variant data generated by LLM. Subsequently, the framework designs an optimization loop, which includes three core steps: mutation, evaluation, and selection, to iteratively optimize the initial prompt. Prochemy is a plug and play solution that can integrate with existing methods such as COT and multi-agent systems without the need to modify their underlying architecture.
IEEE Transactions on Software Engineering (TSE) is a top journal in the field of software engineering, recommended as an A-class journal by the Chinese Computer Society (CCF) and a SCI TOP journal in Chinese Academy of Sciences. The first author of this paper, Ye Sixiang, is a 2024 graduate student of the Institute of Information Technology, under the guidance of Professor Li Zheng and Liu Yong. Beijing University of Chemical Technology is the first unit to complete this paper, and the cooperation units include the Software Institute of the Chinese Academy of Sciences and Peking University. This work is supported by projects such as the National Natural Science Foundation of China and the Huawei HuYangLin Fund.
Downloading paper：https://arxiv.org/abs/2503.11085