DataSync V2

DataSync

1. DataSync will backup capability/report/event data and re-send missing data after connection recover.
   • 
2. The data will backup with SQLite3 (datasync.db).
   • Capability is saved in table TB_Cap.
   • Report and Event data are saved in TB_Data.
3. DataSync will re-send capability first before the missing report data.
4. DataSync always backup data whether connected or disconnected.
5. The recover data and report data can send in the same time.
6. DataSync doesn't monitor network adapter and pythical layer network status, the disconnected status is reported from mosquitto library (software connection status, ex: TCP connection lost..).

How to Verify

You can verify DataSync function by one of following:

1. Make a disconnect event by pull out network wire.
2. Disable network adapter
3. Modify routing table to block WISE-Cloud IP.

Recover Format

DataSync will re-send report data (no event data) with batch format, the format is following:

{  
   "content":{  // "content" is added by SAManager, handler doesn't need add this header
      "data_list":[  
         {  
            "TestHandler":{  // handler report data 1
				...
            }
         },
         {  
            "TestHandler":{  // handler report data 2
				...
            }
         },
         {  
            "TestHandler":{  // handler report data 3
               ...
            }
         }
      ]
   },
   "commCmd":2055,
   "agentID":"00000001-0000-0000-0000-000C2982D9F6",
   "handlerName":"general",
   "sendTS":{  
      "$date":1533008028483
   }
}

DataSync.ini

• File example:

[DataSync]
; database size limit MB
Storage=20
; database duration limit HR
Duration=1
; keep previous data before # SEC.
RetainDWTP=60
; # item in each batch
BatchSize=3
; time between each wrap batch send, SEC.
WrapSendInterval=3
; preserved storage space. MB
PreserveStorage=200

• Detail:
  • Storage
    • A limit of database file size, the database will re-cycle after file size large than this limit.
  • Duration
    • A limit of database live time, the database will re-cycle after agent booting time large than this limit.
  • RetainDWTP
    • There is a different between disconnected time and disconnected detected time, so DataSync will re-send the data from "disconnected detected time - RetainDWTP" to recover full data.
  • BatchSize
    • DataSync will send missing data with batch format, it can customize the number of data in one batch.
  • WrapSendInterval
    • The time interval between each wrapped batch data.
  • PreserveStorage
    • DataSync will guarantee the free space is large than "PreserveStorage", database will re-cycle in this condition.

• Priority:
  • There are 3 limit related config, Storage, Duration, PreserveStorage, if user config all, DataSync has a priority for this condition.

PreserveStorage > Storage > Duration Note: You must config at least one of 3 limit, or DataSync will report error.