Hello Vladimir Medvedev,
Any implementation of additional protocol can be done by writting a custom ThingWorx Extension. More on that:
Especially take note about Authenticators - entities that are responsible for user authentication:
Implementing OAuth could also require to create a custom Authentication Filter for Tomcat.
Hope that above information helps you.