You might want to look at Side-by-Side manifest maker. It will do the work for you so that you don't have to stress over it.
If you're not going to do that, then the manifest you need to create two manifests. One is for the assembly itself and will contain the registry entries that you are no longer putting in the registry. You can get the CLSID for that from those entries, but it shouldn't be any different that what you were using when you were using the registry. This manifest has to have the same name as the assembly, with a .manifest extension added (e.g., myassembly.dll.manifest).
The other manifest file is for the executable that is using the assembly, since I'm assuming you won't be adding the assembly to the GAC. That manifest file will contain the reference to the side-by-side assembly.
The way you manually embed manifests in executables and assemblies is with the Manifest Tool that is part of the Windows SDK. Note that you can't run that on PowerBuilder Classic executables, because PowerBuilder executables are non standard. They contain extra information after the standard executable file format that tools like the Manifest Tool will truncate.
For a PowerBuilder Classic executable, you want to make sure that new style controls are *not* selected in the application project because that will create an internal manifest. Instead, embed that manifest information in the external manifest file that you'll put along side the PowerBuilder executable.
The reason is that Windows will only use one manifest file for an executable, and the behavior varies by the version of Windows. Some will ignore an external manifest if it sees an internal, and some will ignore an internal if it sees an external. If you only have an external, you don't have to worry about it.
As I mentioned at the beginning of all this though, that third party tool will do all the work for you. If you're not real comfortable working with manifests, it will save you a lot of heartache. I am rather familiar with manifest, and I still prefer to use that third party tool. It's just a lot easier.