Hi Sivaram, which document are you following to add the custom app ? Is it "Building Custom Integration for PTC Navigate" ? This document was created for 1.2 and was never officially released. Not sure if it applicable to later releases.
Normally you can specify an "AppsCollectionTailoringMashup" when building the AppCollectionRow from your custom AppCollectionProvider::GetAppCollections() implementation. But you will not be able to re-use, as it, the out of the box AppsCollectionTailoringMashup and related AppsCollectionAccessAppConfig used by the part or document apps collections.
Starting with Navigate 1.5, there is apparently a new technique (more configuration based) to register custom apps (Add a Custom Task to Your Homepage).