I had the same problem with QC 11 Starter Edition in a virtualized environment where the IP address could change. I'm not sure why it picks up the IP address instead of machine name during the install, but here is the way I fixed the issue. The steps below replace the hard-coded IP address with 'localhost'. Be warned that you'll be editing the registry and manually updating database tables, so a couple of important notes before I begin:

1. This worked for me, but may not work for you. Perform these steps at your own risk as they're probably not HP-approved.
2. Make sure you have current backups of everything before starting, in case something goes wrong.
3. Any existing projects will be inaccessible after following these steps, so export them to .qcp files and delete the existing projects before you start.

----------

1. Stop the QC Service
2. Edit C:Documents and SettingsAll UsersApplication DataHPALMapplication20qcbin.warWEB-INFsiteadmin.xml, changing hard-coded IP address to 'localhost'. I used Notepad to edit it and noticed some binary data in there, so only change the IP address and don't touch anything else.
3. Open the SQL Server Management Studio
4. Go to the qcsiteadmin_db and execute 3 SQL queries:
a. SELECT * from td.DBSERVERS - This will display the table row that needs to be changed
b. UPDATE td.dbservers
SET DBSERVER_NAME = 'localhost'
WHERE DBSERVER_ID = 1
c. UPDATE td.dbservers
SET DB_CONNSTR_FORMAT = 'jdbc:mercury:sqlserver://localhost:1433'
WHERE DBSERVER_ID = 1
5. Stop the 'SQL Server (MSSQLSERVER)' service
6. Run regedit
* Either do a complete search on the full IP address, or just change 2 entries:
o [HKEY_LOCAL_MACHINESOFTWAREMicrosoftMicrosoft SQL ServerMSSQL.1MSSQLServerSuperSocketNetLibTcp] There should be 3 keys displayed - IP1 is set to the bad IP, IP2 is set to 127.0.0.1 and IPAll. The easiest thing to do is delete the IP1 key family entirely and rename IP2 to IP1.
o [HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionRun] - the QC Tray Icon key has the IP address embedded in it. Change the IP address to 'localhost', leaving the rest of the string as-is..
7. Restart the 'SQL Server (MSSQLSERVER)' service
8. Execute the ALM Server Deployment Wizard from the Start Menu
9. Check to be sure everything still works.
10. Reboot the machine and check again.