Yes to your question, you will have to manually edit the string for each project. However while you can edit the dbid to go about this, you can also instead just edit the connection string in site admin. NOTE: This will require each proejct to be deactivated when you change the string.
My suggestion would be to create a project list in excel and have a row for each project name. Have your new string in place there. This is a good check off method to ensure you don't miss any projects for edit.
In site admin, click on the Site Projects tab to see the full list of domains and projects. Click on a particular project to see the Project Details. Under this tab is the Connection string, which will match what you have: jdbc:mercury:sqlserver. Click on the blue Connection String to edit. You will be prompted to deactivate the project if it is not already. In new pop up window, you can change the string and it has a Test Connection to ensure it can communicate with the database. Click OK when done and your new string is present.
Many though as you mention prefer to edit the dbid.xml as best practice gives way to being able to backup each dbid.xml file as a precaution. That way you have the original settings in case of issue. The direct string edit can be copied too, so whichever method you choose. Deactivation or remove and restore with dbid would be needed as well to ''reload'' the project once the edits are done.