Event List and Event Details

Welcome!

The Smarter AI Platform is an enablement software platform for AI cameras that see, listen, and understand.

The Smarter AI Platform is comprised of:

cameras / camera libraries
cloud servers / services
mobile viewer apps / libraries

The Smarter AI platform supports event triggers based on any combination of:

inferencing jobs and workflows
sensor data
telematics data

This guide will show you how to integrate event lists and event details into your web applications via Smarter AI server/service REST APIs.

Event List and Details

Here is an illustration of how Events and Event Details can be represented in an Event Dashboard. You can see an Event list example on the left and an example of an Event Details view on the right.

Step 1. Getting Ready

1.1. Here is what's needed to begin:

Tenant Name → Your tenant is your private working space for users, cameras, events, data, & more.
API Key → The key to access your tenant through Smarter AI REST APIs.

Please contact [email protected], if you're missing your Tenant Name or an API Key.

1.2. Please ensure that your cameras are up to date with the latest App Firmware, System Image, OEM Image, etc.

Step 2. Get the list of Events

In this section, we'll be discussing:

How to get the list of cameras.
How to get the list of Events for a specific camera or for all cameras in your tenant.

2.1. Retrieve the list of all cameras in your tenant. This list will provide the camera ID number for each camera.
This API has several parameters:

type - Endpoint type:
- APP - Mobile application client.
- DEVICE - A dashcam, a gateway, etc.
status - Device or app status.
- ACTIVE - A device (camera) is onboarded and operational.
- RESET - A device (camera) has been removed from a tenant.

curl --request GET \
     --url https://api.smarterai.com/v4/endpoints?type=device&status=active \
     --header 'Accept: application/json' \
     --header 'SAI-Key: YOUR_API_KEY'

[
  {
    "userId": [USERID],
    "status": "ACTIVE",
    "id": [ENDPOINTID],
    "type": "DEVICE"
  },
  
...

  {
    "userId": [USERID],
    "status": "ACTIVE",
    "id": [ENDPOINTID],
    "type": "DEVICE"
  }
]

The DEVICE userId is required to get a device's (camera) user name.

2.2. Get the device (camera ) user's details (userId).

curl --request GET \
     --url https://api.smarterai.com/v4/endpoints/[ENDPOINTID]/acl \
     --header 'Accept: application/json' \
     --header 'SAI-Key: YOUR_API_KEY'

[
  {
    "userId": [USERID],
    "name": "User",
    "email": "[email protected]",
    "role": "OWNER",
    "imageUrl": "IMG_URL"
  }
]

Multiple users can access the same device (camera) with similar or different roles.

2.3. Get events' count.

curl --head \
     --url https://api.smarterai.com/v4/events \
     --header 'Accept: application/json' \
     --header 'SAI-Key: YOUR_API_KEY'

HTTP/1.1 200 
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
X-Total-Count: 131
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Frame-Options: DENY
Set-Cookie: JSESSIONID=D458E0E3F96F5326C8805DB13C79A7DD; Path=/; Secure; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 05 Aug 2022 11:11:10 GMT

In our example, the X-Total-Count value is 131.

2.4. Get an Event List for a device (camera). You can set the following parameters as well:

endpointId
device id
startTime & endTime (define the time range)

curl --request GET \
     --url https://api.smarterai.com/v4/events?endpointId=[ENDPOINTID]&startTime=1658217600000&endTime=1658476800000 \
     --header 'Accept: application/json' \
     --header 'SAI-Key: YOUR_API_KEY'

[
  {
    "id": "818690217686328",
    "endpointId": [ENDPOINTID],
    "timestamp": 1659705818041,
    "triggerId": "b63ad4e0-ebb3-456c-9649-0c3d8c463965",
    "triggerName": "My Trigger",
    "deviceLabel": "CAM01020000",
    "snapshot1": "SNAPSHOT_URL1",
    "snapshot2": "SNAPSHOT_URL2"
  },

...

  {
    "id": "818667314111082",
    "endpointId": [ENDPOINTID],
    "timestamp": 1659703526909,
    "triggerId": "b63ad4e0-ebb3-456c-9649-0c3d8c463965",
    "triggerName": "My Trigger",
    "deviceLabel": "CAM01020002",
    "snapshot1": "SNAPSHOT_URL1",
    "snapshot2": "SNAPSHOT_URL2"
  },
]

Here are the key parts of the API response:

triggerName

deviceLabel

snapshot1 → Imager 1 snapshot.

snapshot2 → Imager 2 snapshot.

2.5. Get an Event List for all devices (cameras).

# Array with device IDs (continue array if needed)
deviceIDs=("0000", "0001", ...)
for item in "${deviceIDs[@]}"
do
  url='https://api.smarterai.com/v4/events?endpointId='$item'&startTime=1658217600000&endTime=1658476800000'
  curl --request GET \
     --url $url \
     --header 'Accept: application/json' \
     --header 'SAI-Key: YOUR_API_KEY'

  # Process results
  # ...
done

Here is what is provided by the script above:

Events Count
Video Snapshots
Device Name
User Name
Trigger Name (will be used later)

Event List Dashboard Left Part Decomposition

Step 3. Event Details

Get access to and Event's data & details

3.1. Get the video and sensor data corresponding to an event. This request provides Event related data and information, such as:

Pointers (URLs) to media files (dual and multi-imager support).
Sensors data such as accelerometer, gyroscope, location, decoded vehicle data (CAN bus), etc.

curl --request GET \
     --url https://api.smarterai.com/v4/events/804774584580474 \
     --header 'Accept: application/json' \
     --header 'SAI-Key: YOUR_API_KEY'

{
  "id": "804774584580474",
  "tenantId": "[TENANTID]",
  "endpointId": [ENDPOINTID],
  "deviceId": "CAM01020002",
  "startTimestamp": 1658314255636,
  "endTimestamp": null,
  "sources": [
    "Sensor",
    "Sensor"
  ],
  "triggerId": "2693baea-e4a8-495b-a39f-309f277f5e6d",
  "triggerName": "Right Turn",
  "deviceLabel": "John Doe's Car Camera",
  "userInitiatedEventRequestId": null,
  "snapshot1": "[SNAPSHOT1_URI]",
  "snapshot2": "[SNAPSHOT2_URI]",
  "recordings": [
    {
      "id": 1395712,
      "startTimestamp": 1658314225237,
      "endTimestamp": 1658314258167,
      "source": "sensor",
      "type": "DATA",
      "url": "[RECORD_ID]"
    },
    {
      "id": 1395752,
      "startTimestamp": 1658314258261,
      "endTimestamp": 1658314291179,
      "source": "sensor",
      "type": "DATA",
      "url": "[RECORD_ID]"
    },
    {
      "id": 1395750,
      "startTimestamp": 1658314240324,
      "endTimestamp": 1658314273394,
      "source": "vid_1",
      "type": "VIDEO",
      "url": "[RECORD_ID]"
    },
    {
      "id": 1395784,
      "startTimestamp": 1658314273448,
      "endTimestamp": 1658314306386,
      "source": "vid_1",
      "type": "VIDEO",
      "url": "[RECORD_ID]"
    },
    {
      "id": 1395754,
      "startTimestamp": 1658314240759,
      "endTimestamp": 1658314274093,
      "source": "vid_2",
      "type": "VIDEO",
      "url": "[RECORD_ID]"
    },
    {
      "id": 1395835,
      "startTimestamp": 1658314274167,
      "endTimestamp": 1658314307297,
      "source": "vid_2",
      "type": "VIDEO",
      "url": "[RECORD_ID]"
    },
    {
      "id": 1395727,
      "startTimestamp": 1658314238364,
      "endTimestamp": 1658314270697,
      "source": "vid_2",
      "type": "VIDEO_META",
      "url": "[RECORD_ID]"
    },
    {
      "id": 1395774,
      "startTimestamp": 1658314271504,
      "endTimestamp": 1658314303516,
      "source": "vid_2",
      "type": "VIDEO_META",
      "url": "[RECORD_ID]"
    }
  ]
}

The API response includes:

Snapshot URLs - snapshots of all imagers.
Data array items with "source: sensor" and "type: DATA" contains sensor data, including:
- Accelerometer,
- Gyroscope,
- Geolocation,
- and more.

In the Smarter AI platform, video is available as clips of equal duration (called recording units). Each Event has several consecutive video clips (recording units).

Data array items with "source": "imager1" or "source": "imager2" and "type": "VIDEO" contains video clips' URLs. There will likely be several recording units.
Example: A 1-minute event with 10 seconds-long recording units will have between 6 and 8 video clips.
Data array items with "source": "imager1" or "source": "imager2" and "type": "VIDEO_META" contains video meta data:
- AI model labels with confidence levels,
- Bounding box coordinates,
- Video resolution.

3.2. Understanding Event data.

Sensor and other kinds of data (CAN bus, location, etc.) are time synchronized with the video clips.
Data temporal resolution is configurable.
- For video clips from a Smarter AI Dashcam, the default temporal resolution is 15Hz. The spatial resolution, as well as the temporal resolution, depend on the hardware capacity as well as the load on this hardware.
- For sensor data: it depends on the sensor hardware and the configured temporal resolution. It varies between 1Hz for your typical positioning system (GNSS) and 400Hz+ for an accelerometer.

2 1658314225981 -0.000046 -0.000443 0.000046
0 1658314225982 -0.003280 0.009997 0.052085
0 1658314225983 -0.030869 -0.004143 0.037036
0 1658314225996 -0.020178 0.013456 -0.058421
0 1658314225996 -0.029841 0.005512 -0.052965
0 1658314225996 0.004620 -0.011817 -0.003578
0 1658314225996 0.044609 0.028508 0.010184
2 1658314225996 0.000061 -0.000443 -0.000473
2 1658314225996 0.000565 -0.000443 -0.000610
2 1658314226007 0.000198 -0.000443 -0.000900
2 1658314226007 -0.000244 -0.000031 -0.000671
3 1658314226039 25.199066 55.272907 1.300000 0.000000 0.000000 2.202271
0 1658314226044 0.027424 -0.019660 0.065564
0 1658314226044 0.026949 -0.019217 0.064528
0 1658314226044 0.039667 -0.011731 0.053471
0 1658314226044 0.013204 0.014147 -0.011868
2 1658314226046 0.001038 0.000534 -0.000839
2 1658314226046 0.001236 0.000427 -0.000473
2 1658314226046 0.000259 0.000473 -0.000473
2 1658314226046 0.000275 0.000168 -0.000473

File Format → (CSV format) ID TS VAL1 … VAL6

ID - data type identifier

0 - Accelerometer

2 - Gyroscope

3 - GPS Data

TS - timestamp

VALx:

for ID 0 - Accelerometer → Ax Ay Az

for ID 2 - Gyroscope → Ax Ay Az

for ID 3 - GPS Data → Latitude Longitude Altitude Speed Bearing Accuracy

[
  [
    {
      "detections": [
        [
          "LEFT_EYE_CONFIDENCE",
          99.32,
          "%"
        ],
        [
          "RIGHT_EYE_CONFIDENCE",
          95.49,
          "%"
        ],
        [
          "LEARNT_NEUTRAL_HEADPOSE_PITCH",
          -10.65,
          "°"
        ],
        [
          "LEARNT_NEUTRAL_HEADPOSE_YAW",
          8.79,
          "°"
        ],
        [
          "LEARNT_NEUTRAL_HEADPOSE_ROLL",
          1.85,
          "°"
        ],
        [
          "HEADPOSE_PITCH",
          -15.11,
          "°"
        ],
        [
          "HEADPOSE_YAW",
          0.08,
          "°"
        ],
        [
          "HEADPOSE_ROLL",
          -1.93,
          "°"
        ]
      ],
      "bbox": [
        709,
        164,
        908,
        359
      ],
      "ts": 1658314238364,
      "dimen": [
        720,
        1280
      ]
    }
  ],

  [ ... ]

]

File Format (JSON format)

detections - array with Labels data (Text, Value, Character Unit)

bbox - size and position of bounding boxes (X1, Y1, X2, Y2)

ts - timestamp

dimen - frame resolution (Height Width)

Here is an example showing how to render video footage, sensor data, and location information in a GUI.

Video Recordings (dual-imager)
Inferencing Output
Sensor Data
Event Type and Event Date
Camera Location

Event List Dashboard Right Part Decomposition

Data Visualization Examples

An intuitive GUI displaying video and sensor data in a clear and concise way helps Fleet operators in their day-to-day operations, especially with the following use cases, such as: