Study of quantum dynamics in the transition from classical stability to chaos