13 Replies Latest reply on Jul 21, 2017 2:49 AM by carlesc RSS
    eklavyasirohi Apprentice

    Async service not able to fetch data table entry

    Hi All,

     

    We are facing unexpected behavior of Async service when called from sync service.

    While calling Async thingworx service, database related operations were not functioning well when called from sync service after logging entry in Data table. When async service tries to fetch the entry from data table using primary ID, no result is getting returned.

     

    Logic details:

    Sync service A adds entry in data table and calls Async service B (pass primary ID of same data table entry) for further processing of request.

     

    But this seems to be working windows platform and issue encountered on linux ONLY.

     

    As per the understanding, this could be due to the way platform handles multi-threading as async execute on separate thread or may be accessing data table before commit operation pf previous update.

     

    Please let us know in case anybody faced this issue and what all options resolutions are available.

      • Re: Async service not able to fetch data table entry
        eklavyasirohi Apprentice

        [Update]

         

        Just to update that we have analyzed further on this issue and tried to reproduce the same issue on local Linux VM using same client VM configuration and things are working fine.

        I am not sure what exactly the issue is and there might be some difference in the installed packages. We are still working on this and update further on this thread if I get some more details.

        • Re: Async service not able to fetch data table entry
          carlesc Ninja

          You are facing what you are saying here:

          "As per the understanding, this could be due to the way platform handles multi-threading as async execute on separate thread or may be accessing data table before commit operation pf previous update."

          As async operation it's executed before you sync process ends ( the transactions start when your services starts and ends when your service ends ).


            • Re: Async service not able to fetch data table entry
              eklavyasirohi Apprentice

              Hello Carles Coll,

               

              On client VM (RHEL 7.2 version, 64 bit arch), using one service I am inserting a data in data table and using async to retrieve the already inserted record using the same ID, I am not able to perform or fetch the data entry made sync service and getting message that record doesn't exist.

              This works well when tested locally on windows platform but facing issue when the same is deployed on Linux VM. My understanding was that this could be due to different platforms multi-threading capability.

               

              But another update provided above as per further analysis, this problem is not due to different platforms and everything is working fine even with linux VM when tested locally. There might be some difference in installed packages on both the Linux VMs.

               

              I am still analyzing this and I am not sure what exactly is the problem. It could be configuration differences.

                • Re: Async service not able to fetch data table entry
                  carlesc Ninja

                  You never know when a async process will be called ( at the same time, after, sometimes at the same time some times after,..), then it may act differently depending on the platform, but you should avoid this kind of situation, it's a shame but you will need to execute it synchronously or put on a queue ( there's no queues system on TW you will have to build your own or import a queue system as an extension ).

                    • Re: Async service not able to fetch data table entry
                      eklavyasirohi Apprentice

                      Synchronous execution is not the way because we expect large processing and to avoid long wait time for client.

                      Also, we are using database (Data table) queue for scheduler to handle failed requests.

                        • Re: Async service not able to fetch data table entry
                          carlesc Ninja

                          Then go for the queue solution ( we also have a DataTable/Queue system ).

                            • Re: Async service not able to fetch data table entry
                              eklavyasirohi Apprentice

                              I am not sure how queue solution will work here then !!

                              Could you please provide some details how this solution will solve the problem.

                              • Re: Async service not able to fetch data table entry
                                jacekgra Apprentice

                                Hello Carles,

                                 

                                What "queues" do you have in mind? I am wondering whether it is possible to queue DataTable services execution (which are coming from multiple Things from multiple threads)?

                                I am facing an issue: I have multithreaded SDK Agent which push data to ThingWorx for ~15 devices every 0.5sec (a lot of remote properties) and very, very complex logic for subscription which is also triggered by the Agent whenever there is an update of the properties - in this logic I am interacting with the datatables (especially one) multipletimes (Queries, AddOrUpdate, etc) which causes data inconsistency (as multiple threads are accessing data which may be updated in the meantime by other thread and moreover - sometimes PersistenceProvider is not able to handle the data and I have an error from it).

                                I know that I should go most likely for external DB, but I would like to consider possibilities in TWX (DataTables OOTB Queue mechanism for DataTables seems very interesting).

                                 

                                Twx 7.3.5 (PostgreSQL).

                                 

                                Thanks in advance for any hint.

                                 

                                Regards,

                                Jacek

                                  • Re: Async service not able to fetch data table entry
                                    carlesc Ninja

                                    Hi Jacek,

                                     

                                    When I mean DataTable/Queue, I mean a custom code which uses DataTables + Timers to build a queue:

                                     

                                    • DataTable with Service Calls
                                    • Timer which consumes Service Calls on DataTable

                                     

                                    Then you control concurrency on subscription calls, the subscription to data change event just adds a Pending Service Call tot he DataTable, and then the consumer ( Timer ) consumes the queue.

                                     

                                    TW should have Queues out of the box...

                                     

                                    We are using it effectively on our production server, for instance we have a Delayed Data Change Event, which automatically adds the Data Change Event to the queue for execution later on. We also use it for instance to Iterate over a large amount of long running process in order to cut transactions in small pieces, as the worst thing that can happen to TW it's a long running service ( aka > 10minutes )

                                     

                                    Best Regards,

                                    Carles.

                        • Re: Async service not able to fetch data table entry
                          eklavyasirohi Apprentice

                          Just a workaround,

                          We have added pause function (seconds of delay) on async thread to avoid race condition for time being. Ideally, there should be only one triggering point for async call but we designed it as per our use case requirement. We are still looking for better implementation and might re-design this solution accordingly.

                           

                          We have dona a good amount of analysis for this and anyone can contact if required.

                            • Re: Async service not able to fetch data table entry
                              carlesc Ninja

                              Hi Eklavya,

                               

                              Not, with pause you don't solve anything, you will face exactly the same problem, as your async process starts at the same time as the other transaction it's running, then on the starting point of the Async process the Database status it's without data.

                               

                              You don't control the transaction, the transaction it's controlled by TW and it starts at Service Call.

                               

                              Carles.