@jdcast Just ran into this myself, I had to reflash (there may have been a way to fix it without that but for me reflashing was easier). Once your system is in a working state, make this file to pin the packages until the driver is fixed. Once it’s fixed you should just be able to delete this file:
sudo vim /etc/apt/preferences.d/jetson-l4t-pinning
Contents:
Package: nvidia-l4t-*
Pin: version 36.4.3*
Pin-Priority: 1001
Package: nvidia-jetpack*
Pin: version 6.2+b77
Pin-Priority: 1001
You should be free to run update/upgrade without issues with this file in place. Note I was flashing jetpack 6.2, the nvidia-jetpack version might be different if you are installing 6.1, etc. But you can check it with apt search nvidia-jetson and just pin it to whatever it currently is