The first thing that I would suggest for an upgrade that continues to fail is to turn on performance monitors for the server machine, so that you can see what is occurring when the upgrade crashes. I would suggest anytime you perform an upgrade to set the Performance Monitor in windows and pay special attention to the memory usage during the upgrade process. The bigger the project the bigger the memory will need to be on the QC server machine to handle the upgrade.

9/10 times if an upgrade is failing at different spots every time during the process then it is some kind of lack of memory, whether JBOSS or regular system memory.

When the upgrade errors out, are you able to log back into the server directly after the event, or is the server muted?
If your server is muted then it could be due to the JBOSS heap size being too small when the upgrade is performed, sometimes on super large projects like yours I have seen the QC server muted for exceeding the JBOSS heap. Another thing to mention here is that you will also need to monitor performance on your database server machine as well, since both the database server and the QC server are both accessed during project upgrades from QC10 both of these machines need to have sufficient available memory when the upgrade process is being run.

Also make sure to run the verify and repair and manually fix any errors that require manual intervention before the upgrade process so that you are sure that the project is ready for the upgrade to v11.