5 Replies Latest reply on Aug 16, 2017 3:07 PM by asmith RSS
    asmith Newbie

    Successful TW Foundation connection writes errors in cxserver.log

    When I start the TW Azure IoT Hub Connector I see both an AzureIot cxserver Remote Things and a Connection Server shown on the TW Foundation server monitoring mashup. I am also able to browse to the TW Azure IoT Hub Connector at port 9009 and it returns "Connected to the platform"

     

    However, when I check the cxserver.log file I see the file contents shown below. Is everything working, or do I have a problem?

     

    15:44:51.145 [main] WARN  c.t.c.p.s.k.impl.StoreServiceImpl - The contents of cache: ThingNameStash will never expire: -1. Consider setting 'default-expire-time' or the cache 'expire-time'.

    15:44:51.418 [main] WARN  c.t.c.metrics.MetricsReporterManager - Metrics are enabled but no reporters are configured

    15:44:54.632 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: Exception while initializing stores, not starting partition manager

    15:44:54.632 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: Caught java.util.concurrent.ExecutionException: com.microsoft.azure.storage.StorageException: The server encountered an unknown failure:

    15:44:54.632 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.util.concurrent.FutureTask.report(FutureTask.java:122)

    15:44:54.632 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.util.concurrent.FutureTask.get(FutureTask.java:192)

    15:44:54.632 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.eventprocessorhost.PartitionManager.initializeStores(PartitionManager.java:202)

    15:44:54.632 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.eventprocessorhost.PartitionManager.initialize(PartitionManager.java:106)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.eventprocessorhost.EventProcessorHost.lambda$registerEventProcessorFactory$0(EventProcessorHost.java:509)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.util.concurrent.FutureTask.run(FutureTask.java:266)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.lang.Thread.run(Thread.java:748)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: Inner exception com.microsoft.azure.storage.StorageException: The server encountered an unknown failure:

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.storage.StorageException.translateException(StorageException.java:101)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:199)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:769)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:756)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:730)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: com.microsoft.azure.eventprocessorhost.AzureStorageCheckpointLeaseManager.lambda$leaseStoreExists$4(AzureStorageCheckpointLeaseManager.java:236)

    15:44:54.633 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.util.concurrent.FutureTask.run(FutureTask.java:266)

    15:44:54.634 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

    15:44:54.634 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

    15:44:54.634 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-cf64d03e-26d6-4a2c-84c5-068da86de117: java.lang.Thread.run(Thread.java:748)

      • Re: Successful TW Foundation connection writes errors in cxserver.log
        ekobyakov-2 Explorer

        this indicates a problem with EventHub stream checkpoint storage. It is usually one of two things:

        - storage account is not configured correctly

        - the storage information is out of date with the state of event hub stream it may happen if event/iot hub was re-created but storage was not cleaned up

         

        For background: Azure library that we use to process event hub streams is using azure storage to store stream position information and manage consumer leases. Connector creates blob storage container named eph-storage-${iot-hub-name} in the configured storage account.

          • Re: Successful TW Foundation connection writes errors in cxserver.log
            asmith Newbie

            Are there any other requirements for setting-up the Azure side beyond an IoT Hub and a blob storage account before I start the TW Connector?

             

            I have an empty IoT Hub an an empty blob storage account. I have tried deleting and recreating the Azure components and I always get the same error.

             

            I never see a container created by the connector. I tried creating the container named like you mentioned above (eph-storage-twdev) and I see the same error:

             

            19:49:02.812 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-5db49cf1-da51-4744-8872-621cae709dc2: Exception while initializing stores, not starting partition manager

            19:49:02.812 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-5db49cf1-da51-4744-8872-621cae709dc2: Caught java.util.concurrent.ExecutionException: com.microsoft.azure.storage.StorageException: The server encountered an unknown failure:

            19:49:02.812 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-5db49cf1-da51-4744-8872-621cae709dc2: java.util.concurrent.FutureTask.report(FutureTask.java:122)

            19:49:02.812 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-5db49cf1-da51-4744-8872-621cae709dc2: java.util.concurrent.FutureTask.get(FutureTask.java:192)

            19:49:02.812 [pool-5-thread-1] ERROR eventprocessorhost.trace - host AzureIot-5db49cf1-da51-4744-8872-621cae709dc2: com.microsoft.azure.eventprocessorhost.PartitionManager.initializeStores(PartitionManager.java:202)

             

             

             

            Here are the blob store settings from Azure - do they look correct?

             

             

             

             

             

             

             

             

            ...

          • Re: Successful TW Foundation connection writes errors in cxserver.log
            asmith Newbie

            I upgraded Java from openjdk version "1.8.0_141" to Oracle Java(TM) SE Runtime Environment (build 1.8.0_144-b01) and it started working.

             

            It seems the TW Azure Connector requires official Oracle Java, I could not get openjdk to work.