Time series prediction using datetime type as temporal

[precisionlabs]

I uploaded a time series dataset setting the temporal op type to a field containing a ISO parsed datetime and setting the type of this column as a DateTime. I took caution to set the time sampling interval to the correct number of miliseconds (every row was a 1 minute delta so samplig was 60,000)

I then trained a time series model with this dataset. No issues there.

Now when I try to get a prediction I get the following message:

Error executing service predWaterFlowDatetime Message :: Could not process request: [For input string: "2016-03-08T02:00:00.000Z"

I'm basing my code and data on this tutorial: https://community.ptc.com/t5/IoT-Tips/TWX-Analytics-How-to-Build-and-Operationalize-a-Time-Series/ta-p/818275

In this tutorial they use a simple integer as the temporal field, I got that working no problem. I've now adapted the code, datashape, dataset metadata and model to use this datetime typed column as the temporal field, but something is not working. I've tried changing a lot a things to no avail.

That error message is the closest I've gotten. It appears that the data I'm sending to Analytics for the prediction is typed as a string and it is not able to parse it. Or simply Analytics doesn't have this capability, which would be weird considering it managed to train a model using a DateTime type.

I'm out of ideas, has anyone managed to make time series predictions using DateTime data as temporal field?

[Rocko]

I have never tried this myself, so I can't tell if this works at all with DateTime. But did you notice the timestamp in the screenshot of getdatatableentries does not end with the "Z" like the error message does? I wonder where that comes from. Maybe you need to do either different date transformations or use different types when you provide the scoring data.

[precisionlabs]

It's because in the datatable it is correctly formatted as a datetime object. The string that ends with Z and has a T between the data and time, is the ISO formatted string representation of the date.

After reading @cmorfin's reply I'm thinking the datatable is giving me the right data type, but when passing that to Analytics it is parsed as a string and Analytics doesn't know what to do with that. I'm gonna implement what they suggest.

Thanks for your input.

[cmorfin]

Hi

By default the date string that you pass for scoring will be seen as a STRING, hence the error.

You need to also pass the metadata to the scoring service so it knows the date string is a DateTime type.

Taking the example from the post your referenced in the predWaterFlow service, you need to:

- create a new infotable that represent the metadata.

- pass this metadata into the dataset, so basically line 29 you'll add newEntry.metadata = metadata (or whatever name you gave to the metadata info table)

That should help

[precisionlabs]

Let me see if I got this straight. I'm gonna paste the relevant code below.

var dataset = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape({
infoTableName : "InfoTable",
dataShapeName : "AnalyticsDatasetRef"});

var newEntry = new Object();
newEntry.data = datasetRef; 
dataset.AddRow(newEntry);

var predictiveScores = Things["VM-THINGWORX-AnalyticsServer_PredictionThing"].RealtimeScore({
modelUri: "results:/models/" + Model_Guid,
datasetRef: dataset});

The infotable "dataset" has the datashape "AnalyticsDatasetRef" which has a field called "metadata". This "medata" field is created with a datashape called "AnalyticsDatasetMetadata" whose fields corresponds to the metadata entered when uploading the dataset (fieldName, dataType, opType, etc.).

So now I need to make a metadata infotable with the corresponding information, add it to the "newEntry" object, which will add it to the "dataset" infotable so it will be passed to Analytics for scoring.

I'm gonna try this and report back. Thank you so much @cmorfin.

[cmorfin · Accepted solution]

Yes your understanding is correct.

Note that I have created an article to document this, so it may be a little clearer with some example: https://www.ptc.com/en/support/article/CS415472 .

I did run some tests and can confirm it works with this way.

Christophe

[precisionlabs]

It worked!

Thanks for your time @cmorfin, I didn't expect an answer so fast! I have another time series scoring question, but I'm gonna open a new thread.

[cmorfin]

Brilliant !

I am glad that it worked.