9 Replies Latest reply on Mar 17, 2016 8:11 AM by pchung RSS
    eklavyasirohi Apprentice

    Value Streams Vs Data Tables

    As we know, Value Streams are for continuous data streams and Data tables are moreover designed for static data.

     

    What would be an ideal use-case when one should use value streams and Data tables?

      • Re: Value Streams Vs Data Tables
        paic Collaborator

        Not to confuse the picture.

        but there are Streams, ValueStreams and DataTables

        From the Digital Guide:

        A ThingWorx stream is a list of activities from things or data associated with things. Data associated with things includes time series data and event-driven data.

        ValueStreams are also like streams, but they will store the data 'flat', because it does it in an automated fashion, while you have to script how to store the information into a regular stream.

        Streams also work in an asynchronous way and are more suited for large number of entries.

         

        A ThingWorx data table is similar to a Relational Database (RDBMS) Table. A ThingWorx data shape defines the columns or fields of the data table. To use a data shape to define a data table, at least one of the fields in the data shape must be set as a primary key. This enables the update and delete functionality, as well as efficient querying of data.

        DataTables are synchronous and support primary keys and indexing and are more suited for reference/lookup.

         

        So generally anytime you are storing information over time, historical information, think Stream/ValueStream.

        If you need a more static reference table, think DataTable.

        • Re: Value Streams Vs Data Tables
          sponomarenko-3 Explorer

          Hi, everybody

          And what would you recommend to use in following case?

          I have to store SUM value from the number of Things according to 5 mins intervals.

          I've created a simple service which gets the value history from the ValueStream for the desired period and adds the value and Updates the property finally with new SUM value for the desired period. But due to asynchronous ValueStreams if I call 5 times my service fast I'll get only 2 or 3 values added to the SUM value because data is written to disk in 2 seconds (according to default persistent provider) and the next Query will read the previous SUM value.

          It should update SUM correctly also the periods in history when one of the RemoteThing  get connected again and uploads historical data.

          Would DataTable be more suitable? Or is there other best practice?

          Here is the code:

          ------------------------------------

          // params - value = value to be logged, time = DATETIME of property in real time

          var period = 300000; //5 min in ms

          var periodStart = time.getTime();

          periodStart /= period;

          periodStart = Math.floor(periodStart) * period;

          var periodEnd = periodStart + period - 1;

          periodStart = new Date(periodStart);

          periodEnd = new Date(periodEnd);

           

          var curTotal = 0;

           

          var params = {

            oldestFirst: undefined /* BOOLEAN */,

            maxItems: 1 /* NUMBER */,

            endDate: periodEnd /* DATETIME */,

            query: undefined /* QUERY */,

            startDate: periodStart /* DATETIME */

          };

           

          var res = me.QueryPropertyHistory(params);

          if (res && res.rows && res.rows.length > 0){

              curTotal = res.rows[0].totalConsumption;

          }

           

          curTotal += value;

           

          //now update value

           

          var itDef = {

             infoTableName : "InfoTable",

             dataShapeName : "NamedVTQ"

          };

           

          var itNvtq = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(itDef);

          // NamedVTQ entry object

          var newEntry = new Object();

          newEntry.name = "SUM"; // STRING

          newEntry.time = periodStart; // DATETIME

          newEntry.value = curTotal; // VARIANT

          newEntry.quality = "GOOD"; // STRING

          itNvtq.AddRow(newEntry);

           

          me.UpdatePropertyValues({values: itNvtq /* INFOTABLE */}); //use update property to set current value for thing property correctly

          ---------------------------------------------------------------------------------------

           

          While tests in design time I'd executed 5 times immediately with value=1 and current time and got as the result only 2 or 3. But expected to see 5.

            • Re: Value Streams Vs Data Tables
              pchung Collaborator

              I would recommend storing that in a Stream.

              Data/Information over Time you should think Stream (and ValueStream for the logging)

              If it is for reference or lookup think DataTable

                • Re: Value Streams Vs Data Tables
                  sponomarenko-3 Explorer

                  But Stream also works asynchronously. Than means that physical data write will occur after my service is finished and another call of service can read invalid (not stored yet) value.

                  In this case I should have to be sure that write was done and block the execution with some mutex. But I have not found any tools for that. Are there some?

                  Of course I can create special persistent provider with 1 thread, 1 connection and 0 time of waiting before flushing, but I'm not sure that it is the best idea.