When an agile approach fails, then remember rule #0: No dogmas.
Agile is all about being non-dogmatic. If a rule doesn’t work in your case, find something that works.
You need UML? Use it. UML only slows you down? Drop it. UML might have some value for you in some specific cases? Apply it in a smart way.