If you're using llms to refactor, restarting is cheap. Focus on iterating the plan beforehand rather than iterating the code after to make up the gaps in the plan. Your goal should not be to force complete refactor but to iterate the plan until the llm can oneshot green.