14 Replies Latest reply on Mar 8, 2017 6:21 AM by rsingh-3 RSS
    wwang-2 Newbie

    Thingworx and EMS

         Hi,everyboby.I have completed the Raspberry Pi Quickstart documentation.

         I'm able to use EMS to send raspberry voltage and other data uploaded to the thingworx platform.   Now I want to read the data on the sensor that placed on the raspberry through use of EMS,and then  will be the sensor data uploaded to the Thingworx platform .

    Can you recommend several learning documents or examples?

    Thanks!

      • Re: Thingworx and EMS
        supandey Collaborator

        Hi Wang, reading data from the sensor won't be that complicated it would require you to use the library for that sensor which if you don't have it already you will have to search/Google it and using that library you'll gather the data and push it to ThingWorx. For e.g. i used the REST API together with Python to deliver the payload but you can do the same with EMS, below code collected and pushed the data every 5 seconds from the sensor and then to the ThingWorx.

         

         

        #!/usr/bin/python
        import sys
        import time
        import json
        import logging
        import requests
        import Adafruit_DHT
        
        
        # enabling the logging
        try:
            import http.client as http_client
        except ImportError:
            #Python2
            import httplib as http_client
        http_client.HTTPConnection.debuglevel=1
        
        
        # defining the sensor type and the pin on which the data needs to be collected
        sensor = 11
        pin = 4
        var = 1
        #Webserver URL in TW
        Webhost = "http://tw713:8080/Thingworx/Things/TempAndHumThing/Properties/*"
        App_Key = "<appKey>" 
        
        
        while var == 1:
        
         hum, temp = Adafruit_DHT.read_retry(sensor, pin)
         print('Temp={0:0.1f} *C  Hum={1:0.1f}%'.format(temp, hum))
         #ThingName = "TempAndHumThing"
         headers = {'Content-Type': 'application/json', 'appKey': App_Key}
         payload = {'temp': temp, 'hum': hum}
        
         response = requests.put(Webhost, headers=headers,json=payload, verify=True)
          
         time.sleep(5) # sleep for 5 seconds
        

         

         

        There's another great reference/guide you could use from one of the community Blog post Using the C SDK to Deliver Data to ThingWorx from a Raspberry PI

         

        Hope this would help.

          • Re: Thingworx and EMS
            wwang-2 Newbie

            Hi,Sushant

                Thanks for your help!

                I want to send the data of the read sensor to the thingworx platform via EMS this connection.Can I use lua to read sensor data? Now I have speed sensors, ultrasonic sensors and so on. I want to measure the displacement of the ultrasonic sensor through the EMS upload to the platform for observation.What language is used to read the data on the sensor is better? And then in the Template folder under the Template.lua set properties and services.


            • Re: Thingworx and EMS
              wwang-2 Newbie

              Hi,Sushant,

                   About Edge MicroServer(EMS)&LuaScriptResource(LUA) have more information or helpful examples for?

               

              With your best regards!

                • Re: Thingworx and EMS
                  supandey Collaborator

                  Wang, reading through your responses i feel you are looking for more examples on how lua can read your sensor data directly, is that correct? BTW just to put it out here so you are aware that LSR is optional with EMS, you can use EMS standalone too. I'm not aware if we have any examples on how to read the data from your specific sensor using Lua. Have you already tried searching for the available libraries for your sensor that can be easily integrated with lua?

                   

                  Could you share the exact details of your sensor like the model, type etc.. which you are hooking up to Arduino board?

              • Re: Thingworx and EMS
                dbujoreanu Newbie

                Hi Wang,

                 

                Using the Sense HAT is not limited or specific to Thingworx, basically once you have your Pi and Sense HAT hooked and configured you could use the Sense HAT from anywhere using the existing APIs to read/write to/from it. The Thingworx tutorial on getting the basic information from the Raspberry Pi (i.e. CPU freq&temp) which you've already mentioned as having done successfully can be expanded to read Sense HAT specific sensor information. The key idea is that you can call from your Pi LUA script, i.e. a PiTemplate.lua script to read basic Pi information, other Python scripts that read sensor data from the Sense HAT and then supply thier results back to set Thingworx thing properties values. You can also send from Thingworx strings to be displayed on the Sense HAT LED matrix by defining a remote service on the Pi thing and call it with the string to be displayed. The most common operation however would be to read the Sense HAT's sensor data such as  position, humidity and pressure. I have used these resources to learn about reading the Sense HAT sensor data:

                https://www.raspberrypi.org/learning/getting-started-with-the-sense-hat/

                there is also a specific Astro Pi related page you can refer to:

                https://astro-pi.org/get-involved/program-the-sense-hat/python/

                There is of course a fully fledged API documentation page available for the Sense HAT:

                https://pythonhosted.org/sense-hat/

                 

                What I found most useful were the examples from the getting started with the sense hat. I started from the existing LUA example.lua template and built a Pi specific one to include services and properties to read Sense HAT sensor data.

                 

                 

                Just as an example, my PiTemplate.lua script looks like this:

                 

                module ("templates.PiTemplate", thingworx.template.extend)

                properties.cpu_temperature  =     { baseType="NUMBER",  pushType="ALWAYS", value=0 }
                properties.cpu_freq  =     { baseType="NUMBER",  pushType="ALWAYS", value=0 }
                properties.cpu_volt  =     { baseType="NUMBER",  pushType="ALWAYS", value=0 }

                properties.temperature = {baseType="NUMBER", pushType="ALWAYS", value=0}
                properties.pressure = {baseType="NUMBER", pushType="ALWAYS", value=0}
                properties.humidity = {baseType="NUMBER", pushType="ALWAYS", value=0}

                properties.yaw = {baseType="NUMBER", pushType="ALWAYS", value=0}
                properties.roll = {baseType="NUMBER", pushType="ALWAYS", value=0}
                properties.pitch = {baseType="NUMBER", pushType="ALWAYS", value=0}

                 

                serviceDefinitions.GetSystemProperties(
                output { baseType="BOOLEAN", description="" },
                description { "updates properties" }
                )

                 

                services.GetSystemProperties = function(me, headers, query, data)
                log.trace("[PiTemplate]","########### in GetSystemProperties#############")
                queryHardware()
                queryOrientation()
                queryEnvirons()
                return 200, true
                end

                 

                function queryOrientation()

                local f = io.popen("python /home/pi/twx/microserver/etc/custom/templates/get_pitch.py")
                local s = f:read("*a")
                properties.pitch.value = s
                f:close()
                log.debug("[PiTemplate]", string.format("Pitch: %.2f", properties.pitch.value))

                 

                local f = io.popen("python etc/custom/templates/get_roll.py")
                local s = f:read("*a")
                properties.roll.value = s
                f:close()
                log.debug("[PiTemplate]", string.format("Roll: %.2f", properties.roll.value))


                local f = io.popen("python etc/custom/templates/get_yaw.py")
                local s = f:read("*a")
                properties.yaw.value = s
                f:close()
                log.debug("[PiTemplate]", string.format("Yaw: %.2f", properties.yaw.value))

                 

                end

                 

                 

                function queryEnvirons()

                local f = io.popen("python etc/custom/templates/get_humidity.py")
                local s = f:read("*a")
                properties.humidity.value = s
                f:close()
                log.debug("[PiTemplate]", string.format("Humidity: %5.2f", properties.humidity.value))


                local f = io.popen("python etc/custom/templates/get_pressure.py")
                local s = f:read("*a")
                properties.pressure.value = s
                f:close()
                log.debug("[PiTemplate]", string.format("Pressure: %6.1f", properties.pressure.value))


                local f = io.popen("python etc/custom/templates/get_temperature.py")
                local s = f:read("*a")
                properties.temperature.value = s
                f:close()
                log.debug("[PiTemplate]", string.format("Temperature: %4.1f", properties.temperature.value))


                end

                 

                function queryHardware()
                local tempCmd = io.popen("vcgencmd measure_temp")
                local freqCmd = io.popen("vcgencmd measure_clock arm")
                local voltCmd = io.popen("vcgencmd measure_volts core")
                local s = tempCmd:read("*a")
                s = string.match(s,"temp=(%d+\.%d+)");
                log.debug("[PiTemplate]",string.format("temp %.1f",s))
                properties.cpu_temperature.value = s
                s = freqCmd:read("*a")
                log.debug("[PiTemplate]",string.format("raw freq %s",s))
                s = string.match(s,"frequency%(45%)=(%d+)");
                s = s/1000000
                log.debug("[PiTemplate]",string.format("scaled freq %d",s))
                properties.cpu_freq.value = s
                s = voltCmd:read("*a")
                log.debug("[PiTemplate]",string.format("raw volts %s", s))
                s = string.match(s,"volt=(%d+\.%d+)");
                log.debug("[PiTemplate]",string.format("scaled volts %.1f", s))
                properties.cpu_volt.value = s
                end

                tasks.refreshProperties = function(me)
                log.trace("[PiTemplate]","~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In tasks.refreshProperties~~~~~~~~~~~~~ ")
                queryHardware()
                queryOrientation()
                queryEnvirons()
                end

                 

                The python scripts for doing the actual reading look like this example:

                 

                def get_pitch():

                    from sense_hat import SenseHat

                    sense = SenseHat()
                    sense.clear()

                    o = sense.get_orientation()
                    pitch = round(o["pitch"],1)
                    return pitch
                    
                print("{0}". format(get_pitch()))

                 

                And they all follow a similar approach, defined in the Astro Pi and the Raspberry Pi Sense Hat web pages mentioned above.

                 

                One mention I'd like to make is on how to get location information while using the Pi board. The Sense HAT has no such feature but given that the Raspberry Pi has several USB ports, inserting a USB GPS dongle is pretty straight forward while connecting and reading location information from the dongle is not a trivial task but neither is it very difficult given the amount of resources already available about connecting and reading NMEA data from a USB GPS dongle.

                 

                Hope this info helps.

                 

                Regards,

                 

                Denis

                  • Re: Thingworx and EMS
                    wwang-2 Newbie

                    Hi,Denis

                         Thank you for your helpful assistance.This is a very useful example.

                         If you want to read any of the sensors attached to the raspberry pi , and then send it to the Thingworx platform via EMS and LSR. First you need to read the data on the sensor with the python script, and then set the property/services in the Template.lua.  Is that right?

                         Is it possible to read the data on the sensor directly with lua ,without a python script?




                    Yours sincerely! 

                      • Re: Thingworx and EMS
                        supandey Collaborator

                        Wang, what sort of sensor do you have? I've not done that myself so not sure if you can read it, probably it'll just be about finding the libraries within lua that can do that for you, so that you can directly read from the sensor . Though i feel (may be just my opinion here) python is better equipped with quite a view libraries with useful functionalities. 

                        • Re: Thingworx and EMS
                          dbujoreanu Newbie

                          Hi Wang,

                           

                          As far as I could find, the Sense Hat can be queried using the Python scripts and libraries that come already compiled and packaged for it so if for example you'd attach to the GPIO port some other sensors then you won't necessarily need to use the Python libraries, they may not even exist for your particular sensor type and make, however given that Raspberry Pi does not explicitly prevent you from reading/writing to specific pins on the GPIO using any method, bash scripting or C compiled code for example, I don't see why you should be limited to calling Python from Lua. So far however the preferred approach with Raspberry Pi is to go for the Python scripting and existing libraries.

                           

                          I used this approach of calling Python scripts from Lua since the Sense Hat already had came with the building blocks I needed to interact with the sensors so it didn't make much sense (no pun intended) to reinvent all the libraries that are already written and work quite well for the Sense HAT just so I don't make an outbound call to a Python script and have to read back its results. It actually helped to separate sensor specific code and Thingworx specific code.

                           

                          I've also done some similar work with Arduino, if you're familiar with it, and at least in my opinion the Raspberry Pi and its approach to interact with add-ons and other boards is somehow at a higher level, not as low level as you have for Arduino, which mainly uses embedded C libraries for all of the sensors and other things you can connect to it. The Python scripting that is preferred in Raspberry Pi can be a bit of a put-off at first but once you go past Python's quirks it's easy to get along.

                           

                          One other aspect I see is that  the EMS is biased in the sense that it has to use Lua scripting, the reasons are more or less valid depending on whose point of view you look from, but at least for me, having to call a third party script from Lua and pass parameters then process results is not a major setback. I also wanted to have GPS capabilities for the Pi so I attached a USB GPS dongle to one of my Raspberry Pi USB ports. Since this USB GPS dongle doesn't have any Python scripting already built into the Raspberry Pi libraries I had to devise a way to get the location information from the dongle in some other ways. I ended up using Python as well, not for some specific limitation but since all other scripts were in Python I didn't see a reason why change the approach, so after some digging, I had a script that can be called from Lua and it will return the location as recorded by the GPS dongle. It's not a perfect combination but it's consistent and works quite well.

                           

                          Coming back to the Lua and Python requirement, this I think is of concern for the current setup, i.e. using Raspberry Pi with Sense HAT, however if you're planning to go onto a much broader scope, having Lua as the center point, "standard" language in which all Thingworx platform related code is done is a good approach -  you only need to worry about how your specific sensor/thing/device can be called from the OS and pass on the results or receive input parameters. Also considering that you may have to integrate devices from different vendors using different modes to interact with them makes a "standard" language for the sensors unfeasible.

                           

                          Hope this helps and if you need some extra info about connecting the GPS dongle let me know, I can share the code I used.

                           

                          Regards,

                          Denis

                            • Re: Thingworx and EMS
                              wwang-2 Newbie

                              Hi,Denis,

                                   Thank you for your help.I don't have sense HAT this module and I am using Arduino's development board . Now I want to read the data on the ultrasonic sensor through EMS to the thingworx platform . You also said that EMS can only use lua scripting , I shouldn't use lua scripting to read data?  Can I read it in other languages?


                              Yours sincerely!

                        • Re: Thingworx and EMS
                          rsingh-3 Explorer

                          Hi Wang, Aanjan Ravi,

                          I am trying Raspberry Pi Quickstart documentation and stuck at Step 10(Run LSR). I am using EMS provided in the guide in beginning steps.

                           

                          When i am running LSR it starts but it is giving error as

                           

                          [INFO ] 2017-03-06 17:20:46,870 PiThing: ------------------------------------------------

                           

                          [INFO ] 2017-03-06 17:20:48,838 PiThing: -- Starting script --------------------------

                          [INFO ] 2017-03-06 17:20:48,838 PiThing: Registering core callback handler

                          [INFO ] 2017-03-06 17:20:48,838 PiThing: Starting main loop

                          [INFO ] 2017-03-06 17:20:48,838 PiThing: Calling lifecycle start listeners.

                          [INFO ] 2017-03-06 17:20:48,838 shapes.propsubscribe: Initialized

                          [INFO ] 2017-03-06 17:20:48,839 thingworx.handler: Creating a new handler.

                          [WARN ] 2017-03-06 17:20:48,943 HttpClient::processReq: Authentication requested but no credentials supplied

                          [INFO ] 2017-03-06 17:20:48,944 PiThing: Error occured while accessing EMS. Checking isConnected.

                          [WARN ] 2017-03-06 17:20:49, 46 HttpClient::processReq: Authentication requested but no credentials supplied

                          [INFO ] 2017-03-06 17:20:49, 46 PiThing: MicroServer is now available.

                          [INFO ] 2017-03-06 17:20:49, 46 PiThing: EMS is available: true, online: false

                           

                           

                          config.json

                           

                          {

                          "ws_servers": [{

                          "host": "guragidev01.strykercorp.com",

                          "port": 8081

                          }],

                          "http_server":  {

                              "host": "10.117.255.20",

                              "port": 8081

                          },

                          "appKey": "213abab3-a234-4357-a1f7-49e59d6ac8db",

                          "logger":   {

                              "level":    "DEBUG"

                          },

                          "auto_bind":    [{

                                  "name": "PiThing",

                                  "gateway":  false

                              }],

                          "certificates": {

                              "validate": false,

                              "allow_self_signed":    true

                          },

                          "ws_connection":    {

                              "encryption":   "none",

                              "verbose":  true,

                              "msg_timeout":  1000

                          }

                          }

                           

                          PiTemplate.lua

                           

                           

                          module("templates.PiTemplate", thingworx.template.extend)

                           

                          properties.cpu_temperature = { baseType="NUMBER", pushType="ALWAYS", value=0 }

                          properties.cpu_freq = { baseType="NUMBER", pushType="ALWAYS", value=0 }

                          properties.cpu_volt = { baseType="NUMBER", pushType="ALWAYS", value=0 }

                           

                           

                           

                          serviceDefinitions.GetSystemProperties(

                           

                          output { baseType="BOOLEAN", description="" },

                          description { "updates properties" }

                          )

                          -- service input parameters

                          -- me: table that refers to the Thing

                          -- headers: table of HTTP headers

                          -- query: query parameters from the HTTP request

                          -- data: lua table containing the parameters to the service call.

                          services.GetSystemProperties = function(me, headers, query, data)

                          log.trace("[PiTemplate]","########### in GetSystemProperties#############")

                          queryHardware()

                          --  if properties are successfully updated, return HTTP 200 code with a true service return value

                          return 200, true

                          end

                          function queryHardware()

                          -- use the vcgencmd shell command to get raspberry pi system values and assign to variables

                          -- measure_temp returns value in Celsius

                          -- measure_clock arm returns value in Hertz

                          -- measure_volts returns balue in Volts

                          local tempCmd = io.popen("vcgencmd measure_temp")

                          local freqCmd = io.popen("vcgencmd measure_clock arm")

                          local voltCmd = io.popen("vcgencmd measure_volts core")

                          --   set property temperature

                          local s = tempCmd:read("*a")

                          s = string.match(s,"temp=(%d+\.%d+)");

                          log.debug("[PiTemplate]",string.format("temp %.1f",s))

                          properties.cpu_temperature.value = s

                          --  set property frequency

                          s = freqCmd:read("*a")

                          log.debug("[PiTemplate]",string.format("raw freq %s",s))

                          s = string.match(s,"frequency%(45%)=(%d+)");

                          s = s/1000000

                          log.debug("[PiTemplate]",string.format("scaled freq %d",s))

                          properties.cpu_freq.value = s

                          --  set property volts

                          s = voltCmd:read("*a")

                          log.debug("[PiTemplate]",string.format("raw volts %s", s))

                          s = string.match(s,"volt=(%d+\.%d+)");

                          log.debug("[PiTemplate]",string.format("scaled volts %.1f", s))

                          properties.cpu_volt.value = s

                          end

                          tasks.refreshProperties = function(me)

                          log.trace("[PiTemplate]","~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In tasks.refreshProperties~~~~~~~~~~~~~ ")

                          queryHardware()

                          end

                           

                          Config.lua file

                           

                          scripts.log_level = "INFO"

                           

                          scripts.PiThing = {

                              file = "thing.lua",

                              template = "PiTemplate",

                              scanrate = 1000,

                              taskrate = 3000

                          }

                          scripts.rap_host = "guragidev01.strykercorp.com"

                          scripts.rap_port = 8081

                           

                           

                          Since it worked for you can you please help what is the issue.

                          Is my below code ok?  When i tried using 127.0.0.1 is was not even connecting to EMS.

                           

                          scripts.rap_host = "guragidev01.strykercorp.com"

                          scripts.rap_port = 8081

                           

                          What is difference between WS or Http? I am new to Thingworx so excuse me for this basic question.