Analytics Prediction with time series data

[drichter]

Hi,

I have a time series dataset with the following metadata (json):

[
    {
        "fieldName": "identifier",
        "dataType": "STRING",
        "opType": "ENTITY_ID"
    },
    {
        "fieldName": "A",
        "dataType": "DOUBLE",
        "opType": "CONTINUOUS"
    },
    {
        "fieldName": "B",
        "dataType": "DOUBLE",
        "opType": "CONTINUOUS"
    },
    {
        "fieldName": "C",
        "dataType": "BOOLEAN",
        "opType": "BOOLEAN"
    },
    {
        "fieldName": "time",
        "opType": "DOUBLE",
        "dataType": "TEMPORAL",
        "timeSamplingInterval": 1
    }
]

With this and some rows of data (csv) I create a time series dataset with the analytics builder. Than I create a model with goal "C". If I publish this model (via ThingPredictor) I get an Analyis Model with a weird datashape.

The datashape have some _A-offset......, _B-offset.........., _C-offset........ fields plus a key and a _predicted_C_mo field. In a this Topic https://community.ptc.com/t5/IoT-AR-Tech-Tips/How-to-import-time-series-predictive-models-generated-by/m-p/535191 @jgreiner say I can delete the offset-fields, which means I only have the key and the _predicted_C field. I sounds not correct.

So if I don't change the datashape and try to test it I get the following error unter the "Analytics Jobs": "The dataset must contain entity ID and temporal fields in order to perform TimeSeries training. Please set the timeSeries post body field to null to perform standard training or supply a TimeSeries appropriate dataset. Entity ID field index=-1, temporal field index=-1."

How resolve this problem?

PS: Why is PTC not able to provide meaningful documentation about "how analytics / or prediction work"?

[cmorfin]

Hi

You find the right document https://community.ptc.com/t5/IoT-AR-Tech-Tips/How-to-import-time-series-predictive-models-generated-by/m-p/535191 .

This post says to delete the offset field but it also says to "Edit the DataShape generated by ThingPredictor to match the format of the data that was provided during the model training process"

So in your case you need to delete the offset fields from the DataShape and add the fields used for training in your example: identifier, A,B,C and time.

You should then be able to perform a scoring with ThignPredictor.

Note that you will need to add at least as many row as the lookback window of the model. Possibly the easiest way to find your lookback window is to simply add one row only and execute it. The error message will indicate the value, for example:

Cannot perform a time series transformation when the data count [1] is less than the lookback window size [8].

in this case you will need to enter at least 8 rows of data.

I am taking your point on the documentation , there is definitively something lacking in this area. I will open a case for you to chase this internally.

Hope this helps

Kind regards

Christophe

[skef]

Hello,

thank you very much for clarification regarding editing the input data shape.

I am using a similar file like @drichter. The file contains a continuous field called ttf, which is know while I create the model (the measured ttf of machines) and several other parameters. Now, I want to use the model in order to predict the ttf of machines. I test the model with the input FULL_RANG, the goal ttf, and several other parameters like sensor values of the machine. Depending on the lookback size, I have to use an input of 2 - 8 data rows. But here, of course, the input parameter ttf is not known, since its the goal I want to predict. When I submit the job, I am getting the following error:

The row evaluated to a null score. Unexpected value(s) found in the record. Field [ttf-offset[0]] had value of [null]; Field [ttf-offset[0]-windowfunction[average]-size[2]] had value of [null]; Field [ttf-offset[1]] had value of [null]; Field [ttf-offset[0]-windowfunction[average]-size[4]] had value of [null]; Field [ttf-offset[2]] had value of [null]; Field [ttf-offset[2]-windowfunction[average]-size[2]] had value of [null]; Field [ttf-offset[3]] had value of [null]; Field [ttf-offset[4]] had value of [null]; Field [ttf-offset[0]-windowfunction[average]-size[8]] had value of [null]; Field [ttf-offset[4]-windowfunction[average]-size[2]] had value of [null]; Field [ttf-offset[5]] had value of [null]; Field [ttf-offset[6]] had value of [null]; Field [ttf-offset[4]-windowfunction[average]-size[4]] had value of [null]; Field [ttf-offset[6]-windowfunction[average]-size[2]] had value of [null]; Field [ttf-offset[7]] had value of [null]; Field [ttf-offset[0]-windowfunction[derivative1]-size[8]] had value of [null]; Field [ttf-offset[0]-windowfunction[volatility]-size[8]] had value of [null]; Field [ttf-offset[0]-windowfunction[derivative1]-size[4]] had value of [null]; Field [ttf-offset[4]-windowfunction[derivative1]-size[4]] had value of [null]; Field [ttf-offset[0]-windowfunction[derivative3]-size[8]] had value of [null]; Field [ttf-offset[4]-windowfunction[volatility]-size[4]] had value of [null]; Field [ttf-offset[0]-windowfunction[volatility]-size[4]] had value of [null]; Field [ttf-offset[0]-windowfunction[derivative3]-size[4]] had value of [null];

 This message indicates, that the thing predictor needs the ttf value, but this value is only known when I train the model and could be not a part of the input data shape. Why does this error occurs? What am I doing wrong?

[cmorfin]

Hi @skef

This result is expected.

There are 2 types of time series: the standard time series and the virtual sensor one.

I would recommend you read https://support.ptc.com/help/thingworx_hc/thingworx_analytics_8/#page/thingworx_analytics_8%2Ftwxa-time-series.html as it has got good explanation and diagram that explain what is happening.

In a standard time series, the prediction for the next goal does indeed need the previous goals to be available.

If you do not have them this would probably come into the virtual sensor scenario.

At this time however ThingWorx analytics Builder only supports the standard time series, so you cannot execute  a virtual sensor training in the UI. You will need to use the AnalyticsSerrver_TrainingThing service CreateJob and set the field virtualSensor to true.

Once you have this model you can then score your new data using the AnalyticsServer_PredictionThing service RealtimeScore.

So for the moment using virtual sensor will involve some mashup / service coding from your side.

Hope this help

Kind regards

Christophe

[cmorfin · Accepted solution]

Just to come back on the documentation points mentioned, the Help Center has just been enhanced with some new topics for example about preparing the data for ThingWorx Analytics which I think I quite useful.

Note that regarding ThingPredictor and time series data, this is actually not officially supported and documented at https://support.ptc.com/help/thingworx_hc/thingworx_analytics_8/#page/thingworx_analytics_8%2Fthingpredictor%2Fthingpredictor-limitations.html%23 .

This explains why there is no documentation and that it is cumbersome to configure.

It is expected that in release 8.3 it will be supported with a easier to use configurations steps.

[Leigh]

Hi @drichter

Did Christophe's reply solve your issue? If so, please mark this as an Accepted Solution for the benefit of the rest of our Community. If not, please let us know the current situation.

Thanks!

Leigh

[sarathi]

Hi @drichter did u solve this i am also facing the same issue , If U soveld means please give more details how to resolve that issue,

Am aslo getting same error Like"The dataset must contain entity ID and temporal fields in order to perform TimeSeries training. Please set the timeSeries post body field to null to perform standard training or supply a TimeSeries appropriate dataset. Entity ID field index=-1, temporal field index=-1.