We generally suggest standing up a NEW QC server and DB instance for the NEW load of QC, then migrate the project DB's and repositories over to that environment.
This way, if there is any issue with the upgrade, you can fall back on the original server. QC supports having 11.52 and 12.20 client files loaded on the same PC, so if necessary a project or two could still be addressed on the old server while you troubleshoot the upgrade issue -- usually it is only one or two projects that pose issues on upgrade.

Also, in order to install QC/ALM12.20 on your old 11.52 server, you must UNINSTALL 11.52, leaving QC totally down until you complete the upgrade.
If anything unforeseen happens, you are completely down.

If you are using a virtual machine for the QC server and DB server, you *could* take a snapshot just prior to upgrade activities, then roll back to the snapshots if there is trouble, but that method would cause issues jumping back and forth between old snapshot and new upgrade attempt snapshots. Another approach would be to COPY the VMs, but this also is problematic as you would have duplicate machine names (temporarily) on the network and would need to re-direct the DB server pointers.