cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - Did you know you can set a signature that will be added to all your posts? Set it here! X

Best practise to store historical data plus live streaming data in a Thing

nagrawal-2
1-Newbie

Best practise to store historical data plus live streaming data in a Thing

Hi,

 

What should be the best practice to store the historical data from Excel and then later start storing Live streaming data in same properties of a Thing. I know as per the documentation, I should use Datatables to store static or historical data and Value Streams for Live streaming data but my setup is different. Initially, I need to push some Historical data of a device and later the same device will start pushing live data in TW. I am not sure how would I proceed to achieve this.

 

Please suggest me the best way possible.

Thanks in advance.

 

Best regards,

Neetu

1 ACCEPTED SOLUTION

Accepted Solutions

Hi Carles,

I have resolved the issue. It's not about passing the date-time value in CSV, I had to change the date format in params object:

dateformat: "yyyy-MM-dd HH:mm:ss"

The string should be in exact above format (Case sensitive) to make it work.

Thanks for your support.

Regards,

Neetu

View solution in original post

12 REPLIES 12

If it's the same data that will be recorded live, than all have to go to the same storage, in this case Stream or ValueStream.

Hi Carles,


Thanks for your response.

Yes format of data will be same when recording live. But initially I will be pushing the data from CSV ? Can I push the CSV data in Value Stream?


Regards,

Neetu

Yes for sure, you can read CSV files with the following extensions:

An you can insert data directly on the ValueStream with property owner Thing service:

  • Add+baseType+ValueStreamEntry

We usually combine booth without issues.

Thanks alot Carles, I will try this way.

Regards,

Neetu

Hi Carles,

I am able to read data from CSV file using CSV extension and display it in Mashup in Grid widget. I am not sure how would I push data in ValueStream.

Do I need to create ValueStream and related properties in a Thing?

Regards,

Neetu

Hi Neetu,

Look at my original post the second part:

An you can insert data directly on the ValueStream with property owner Thing service:

  • Add+baseType+ValueStreamEntry

Yes this is what I don't understand. Do you have any example to help me ?

Regards,

Neetu

Hi Carles,

I am able to store the historical data using UpdatePropertyValue service. But when I bind my Property to Time series chart, it is not showing all data values. It is only displaying 30 values out of 260. I don't know what went wrong?

Also, I want to use Date picker widget on my Time series chart so User can see some historical data, would it be possible?

Regards,

Neetu

Hi Neetu,

Did you query Property values through the composer and checked that you have more than 30 values on the history?

Using date picker to query historical data it's not a problem at all, you just need to use startDate and endDate parameters on the property Query services.

I used QueryNumberPropertyHistory and just came to know the data is storing in chunks. I uploaded CSV file with 238 records and only data got saved in ValueStream is 30 records and strangely random data not in ascending or descending order.

I probably be using wrong format of date in CSV

2016-01-31,608,0.0976,20,7.1

2016-02-29,558.0457329,0.1052,18,7.2

2016-03-31,449.9114189,0.08536,22,7.2

2016-04-30,464.1673867,0.0876,25,7.1

I have uploaded so many data in many entities and now realized they all got uploaded wrongly.

Hi,

For what I see on your csv file, you have only Date  and you don't have Time, then if you import that data all the Data at the same day will be overwritten ( I don't know if this is the problem ).

Query one property by one with "Query"+baseType+"PropertyHistory" to see which data it's recorded on each Property

Hi Carles,

I have resolved the issue. It's not about passing the date-time value in CSV, I had to change the date format in params object:

dateformat: "yyyy-MM-dd HH:mm:ss"

The string should be in exact above format (Case sensitive) to make it work.

Thanks for your support.

Regards,

Neetu

Top Tags