Something's broken, but I'm not sure what it is. We do incarcerate a lot of people, but we also have a lot of crime, particularly violent crime, for a developed country. And while we arguably went overboard with strict sentencing, we had a crime epidemic and a lot of overly lenient judges and the truth in sentencing laws did prevent a lot of crime.
If you look at criminal justice reform, there doesn't appear to be nearly as much low hanging fruit as advertised. To meaningfully reduce our incarceration rate, we are going to be letting violent felons out and some of them will commit more violent crimes and we will have a higher crime rate than we otherwise would. Which is fine in theory. We could lower the crime rate substantially by handing out lifetime sentences for any violent crime, but we rightfully aren't willing to make that tradeoff. But it's still a hard sell politically to tell people we're going to purposefully allow the violent crime rate to go up because we want to be more forgiving of people that committed violent crimes previously and in the future.