Camera Onboarding and Pairing

In this section, we will guide you to onboard your camera to the Smarter AI camera platform and pair with the user's account.

Concept

Onboarding
Onboarding is the process of connecting a new device, first to the internet, then subsequently to its enabling services.

Pairing
In IoT, the word Pairing, refers to a process of linking an IoT device with a user account.
This process is also referred as device association.

Understanding the Process

Smarter AI cameras support pairing and onboarding in one go.

To onboard a smarter camera, Internet connectivity needs to be given to the camera.

If the camera supports WiFi, the onboarding process is done through giving WiFI credential to the camera.
If the camera comes with a SIM slot, then it is possible to onboard it over 3G/4G/5G Internet connection.

To pair a camera with a user account, it needs a pairing token and a user ID to be sent to the camera.

Process Flow in Brief

1. App connects with camera

2. Camera sends its onboard state to the app

3. App sends onboarding and pairing credentials to camera

4. Camera uses the onboarding credentials to onboard itself to Smarter AI camera platform

5. Upon successful onboarding, the camera uses the pairing credentials to pair itself with the user account

6. Camera notifies the app about success or fail status over the connection

Connecting With Camera

At first, the Smarter AI camera needs to be turned on. Make sure to use a reliable power source.
If the camera is not onboarded or not paired, it will soon come into pairing mode. When the camera comes into pairing mode, you can connect with the camera to onboard and pair it.

📘

Android

To establish connection with our Smarter AI cameras, you can either use the Android's official documentation of BLE(Bluetooth Low Energy) or use our Smarter AI camera pairing SDK. In this documentation, we will only demonstrate on using the Smarter AI camera pairing SDK to search and establish connection with the camera. To include Smarter AI camera pairing SDK in your project, follow the steps below.

1. Add the following lines in the repositories block of your Project level build.gradle file.

Gradle

    allprojects {
        repositories {
            google()
            jcenter()
            ...
            maven {
                url "http://65.52.189.0:8081/artifactory/gradle-dev-local"
                credentials {
                    username = "username"
                    password = "password"
                }
            }
        }
    }

For sake of security, we share the username and password privately. Contact us here to get the username and password and replace them in the credentials block.

2. Next, add the following line to the dependencies of your module level build.gradle file.

Gradle

implementation 'com.smarter.ai:camera-pairing-sdk:1.0.6'

Replace the version with the latest Smarter AI Camera Pairing SDK version. Contact us here to get the most recent version of Smarter AI Camera Pairing SDK.

3. Next, turn on the bluetooth adapter and location service of your device and add the following permissions in the AndroidManifest.xml file.

XML

    <uses-permission android:name="android.permission.BLUETOOTH" />
    <uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
    <uses-feature android:name="android.hardware.bluetooth_le" android:required="true"/>

4. Finally, take the location permission from the user in runtime while searching or connecting to the camera through the Smarter AI Camera pairing SDK. See the offical documentation of android runtime permission from here.

Finding the Camera Using Bluetooth

To search nearby Smarter AI cameras via bluetooth, invoke scanCameras method from LowEnergyBluetoothScanner. The scanCameras method requires CameraListUpdateListener in the parameter. The newly discovered cameras are returned as a list through the onCamerasUpdated method of CameraListUpdateListener.

Java

BluetoothScanner scanner = LowEnergyBluetoothScanner.getScanner();
scanner.scanCameras(new CameraListUpdateListener() {
    @Override
    public void onCamerasUpdated(List<AnyConnectCamera> cameras) {
        // The newly discovered cameras are returned in the parameter
    }
});

To stop scanning the cameras, invoke forceDiscoveryFinished method of the scanner.

Java

scanner.forceDiscoveryFinished();

Connecting over Bluetooth

The Smarter AI cameras acts as a GATT server and apps should connect to the server as a client. To establish a connection with the camera using Smarter AI Camera pairing SDK, follow the steps below.

1. Get an instance of CameraOnboarder by invoking the getOnboarder() of GattBluetoothCameraOnboarder.

Java

CameraOnboarder cameraOnboarder = GattBluetoothCameraOnboarder.getOnboarder();

2. Then, create an instance of CameraOnboardTask by invoking the onboard method of CameraOnboarder. The onboard method requires an instance of AnyConnectCamera in the parameter.

Java

CameraOnboardTask cameraOnboardTask = cameraOnboarder.onboard(camera)

                                        .withContext(context);

We also need to pass the context by invoking corresponding methods of `CameraOnboardTask`.


3. Next, initialize a callback of `BluetoothConnectionEventListener` and register it to `CameraOnboardTask` to receive bluetooth connectivity events from the camera.

```java
BluetoothConnectionEventListener bluetoothConnectionEventListener = new BluetoothConnectionEventListener() {
    @Override
    public void onConnectionEvent(BluetoothConnectionEvent bluetoothConnectionEvent) {

    }
};

cameraOnboardTask.withBluetoothConnectionListener(bluetoothConnectionEventListener);

The different states of bluetooth connectivity are provided through the onConnectionEvent method of the registered callback. The BluetoothConnectionEvent contains a BluetoothConnectionState and a statusCode. The BluetoothConnectionState defines the connection state of your android device with the camera. This bluetoothConnectionState can be any of the followings.

Connecting : Your device is trying to establish a BLE connection with the camera.

Connected : Your device is successfully connected to the camera.

FailedToConnect : Your device cannot connect to the camera.

Disconnected : Camera has terminated the bluetooth connection with your device.

The statusCode is used to determine the underlying reason of Disconnected state. On the other states, the value of the statusCode is always 0. Click here for more details of disconnection status codes.

4. Next, initialize a callback of GattOnboardEventListener and register it to CameraOnboardTask to receive camera state signals, onboard sdk events and log data from the camera.

Java

GattOnboardEventListener gattOnboardEventListener = new GattOnboardEventListener() {
    @Override
    public void onOnboardCameraEvent(@NonNull GattOnboardEvent event) {
    }

    @Override
    public void onOnboardSdkEvent(@NonNull OnboardSdkEvent sdkEvent) {
    }

    @Override
    public void onReceiveOnboardLog(@NonNull String logData) {
    }
};
cameraOnboardTask.withGattOnboardEventListener(gattOnboardEventListener);

The detailed overview of gattOnboardEventListener is provided later on Reading the Signals from Camera section.

5. Finally, invoke the start() method of CameraOnboardTask to connect with the Smarter AI Camera.

Java

cameraOnboardTask.cameraOnboardTask.start();

📘

iOS

To scan and connect with Smarter AI Camera over bluetooth, use iOSs CoreBluetooth framework and discuss some mentionable methods and their callbacks of this framework. Follow the official guide.

Finding the Camera Using Bluetooth

1. Create a central manager object

An object of CBCentralManager class is the Core Bluetooth object-oriented representation of a local central device.

Objective-c

CBCentralManager *manager = [[CBCentralManager alloc] initWithDelegate:self queue:nil options:@{CBCentralManagerOptionShowPowerAlertKey: @(YES)}];

2. Find nearby Bluetooth devices

To find nearby devices call following method of CBCentralManager class.

Objective-c

[manager scanForPeripheralsWithServices:nil options:nil];

Bluetooth devices will be received through following callback. The newly discovered Bluetooth device is returned as a CBPeripheral object.

Objective-c

- (void)centralManager:(CBCentralManager *)central didDiscoverPeripheral:(CBPeripheral *)peripheral advertisementData:(NSDictionary<NSString *, id> *)advertisementData RSSI:(NSNumber *)RSSI;

Connecting over Bluetooth

The Smarter AI cameras acts as a GATT server. So, you need to connect with peripheral as a Bluetooth client.

Objective-c

[manager connectPeripheral:peripheral options:nil];

If the connection request is successful, the central manager calls the following method of its delegate object.

Objective-c

- (void)centralManager:(CBCentralManager *)central didConnectPeripheral:(CBPeripheral *)peripheral;

You will need some UUID to create a secure communication channel.
For sake of security, we will share the UUID with you privately. Contact us here.

Reading the Signals from Camera

📘

Android

When connection between the camera and your device is successfully established, the camera starts to advertise the Notification State, Onboarding State and Connectivity State to the client through GATT protocol. If you are using our Smarter AI Camera pairing SDK for camera onboarding, you will get these state signals to the onOnboardCameraEvent method of your registered GattOnboardEventListener interface.

Signal Reading and Parsing

The onOnboardCameraEvent method of GattOnboardEventListener gets invoked each and every time when new signal arrives from the camera with a GattOnboardEvent in the parameter. This GattOnboardEvent contains the Notification State, Onboarding State and Connectivity State of the camera.

Sample Code

Java

@Override
    public void onOnboardCameraEvent(@NonNull GattOnboardEvent event) {
    AnyConnectCameraSignal notificationSignal = event.getNotificationSignal();
    CameraOnboardState cameraOnboardState = event.getOnboardState();
    CameraConnectivityState cameraConnectivityState = event.getConnectivityState();
}

The notificationSignal is an instance of AnyConnectCameraSignal which defines different states of the camera while camera onboarding is in progress. The cameraOnboardState is an enum of AnyConnectCameraSignal which represents the onboarding state of the camera. The CameraConnectivityState is also an enum of AnyConnectCameraSignal which indicates the Internet connectivity state of the camera.

📘

iOS

Discovering the Services of a Connected Peripheral

After a connection has been established to a peripheral, discover its available services.

Objective-c

[peripheral discoverServices:nil];

When the specified services are discovered, services for the corresponding peripheral are sent through the following delegate method. Core Bluetooth creates an array of CBService objects, one for each service that is discovered on the peripheral.

Objective-c

- (void)peripheral:(CBPeripheral *)peripheral didDiscoverServices:(NSError *)error {
    for (CBService *service in peripheral.services) {
        NSLog(@"Discovered service %@", service);
        ...
    }

Discovering the Characteristics of a Service

After discovering a service, discover characteristics of the service of the peripheral.

Objective-c

[peripheral discoverCharacteristics:nil forService:interestingService];

The peripheral calls the following method of its delegate object when the characteristics of the specified service are discovered.

Objective-c

- (void)peripheral:(CBPeripheral *)peripheral didDiscoverCharacteristicsForService:(CBService *)service error:(NSError *)error {
    for (CBCharacteristic *characteristic in service.characteristics) {
        NSLog(@"Discovered characteristic %@", characteristic);
        ...
    }

Signal Reading and Parsing

A characteristic contains a single value that represents information about a peripheral’s service. Next, subscribe to get signaling messages from camera.

Objective-c

[peripheral setNotifyValue:YES forCharacteristic:interestingCharacteristic];

You will receive signaling messages from following callback method.

Objective-c

- (void)peripheral:(CBPeripheral *)peripheral didUpdateValueForCharacteristic:(CBCharacteristic *)characteristic error:(NSError *)error {
    int signalingMessage;
    [characteristic.value getBytes: &signalingMessage length: sizeof(signalingMessage)];
}

Depicted Onboarding and Pairing Process Flow

The Camera Signals

After entering into pairing mode, camera starts advertising itself as a Bluetooth Low Energy(BLE) device. In this state, the camera publishes its notification state, onboarding state and connectivity state to the client through a specific characteristic using the GATT protocol. The remainder of a GATT signaling message contains camera diagnostics, which is described in this section.

Notification State Signals

Data Parsing Signals

These signals are sent from camera after receiving onboard credentials from app.

1. INDICATE_PARSING_SUCCEEDED(201)
   Camera has successfully parsed the received onboard credentials.

2. INDICATE_PARSING_FAILED(203)
   Camera could not parse the received credentials.

WiFi Setup Signals

These signals are sent from camera during setting WiFi.

1. INDICATE_WIFI_SET_IN_PROGRESS(301)
   Camera is setting the WiFi credentials received from app.

2. INDICATE_WIFI_SET_SUCCEEDED(302)
   Camera has successfully set up the WiFi received from app.

3. INDICATE_WIFI_SET_FAILED(303)
   Camera has failed set up the WiFi credentials received from app.

Connectivity Check Signals

These signals are sent from camera while checking Internet connectivity.

1. INDICATE_DEVICE_CONNECTIVITY_CHECK_IN_PROGRESS(121)
   Camera is checking if it can reach the Internet.

2. INDICATE_DEVICE_CONNECTIVITY_CHECK_DONE(123)
   Camera has finished checking if it can reach the Internet.

Pairing Signals

These signals are sent while pairing the camera with user account.

1. INDICATE_PAIRING_IN_PROGRESS(501)
   Camera is trying to pair itself with the user account for which the pairing token and user id it has received.

2. INDICATE_PAIRING_SUCCEEDED(502)
   Camera has successfully paired with the user account.

3. INDICATE_PAIRING_FAILED(503)
   The pairing process is unsuccessful.

Other signals

1. INDICATE_DEVICE_ONBOARDED(113)
   Camera is successfully onboarded. In this state, notification signal and onboard state signal give the same value.

2. INDICATE_SERVER_CLOSING(1307)
   Camera has successfully finished with onboard process. Now it is closing the GATT server.

Onboarding State Signals

1. INDICATE_ONBOARDING_NOT_READY(109)
   Camera is not ready to onboard.

2. INDICATE_WAITING_FOR_ONBOARDING(111)
   Camera is expecting onboard and pairing credentials.

3. INDICATE_BUSY_WITH_ONBOARDING(155)
   Camera has received the pairing credentials and currently busy with onboarding process.

4. INDICATE_DEVICE_ONBOARDED(113)
   The camera is onboarded.

Connectivity State Signals

1. INDICATE_CONNECTIVITY_UP(1501)
   Camera can reach the Internet.

2. INDICATE_CONNECTIVITY_DOWN(1505)
   Camera cannot reach the Internet.

Camera Onboarding and Pairing

In this subsection, we will describe step by step camera onboarding and pairing procedure.

Initial Signal Processing Guideline

After getting connected with the camera, you will receive one of 3 types of Onboard state signals.

1. INDICATE_ONBOARDING_NOT_READY(109)
   Camera sends this signal when it has some credentials to onboard itself and trying to do so.
   At this point, you should keep user waiting for the camera to onboard or get ready for onboard.

2. INDICATE_WAITING_FOR_ONBOARDING(111)
   Camera sends this signal when it is not onboarded. So it is waiting for onboard and pairing credentials.
   At this stage, you should ask user for the onboard credentials (WiFi credentials) and request server for pairing credentials through Access library.

3. INDICATE_BUSY_WITH_ONBOARDING(155)
   Camera sends this signal when it has received the pairing credentials and currently trying to onboard using those credentials.
   At this stage, you should show corresponding message to the user based on the Notification state signal

4. INDICATE_DEVICE_ONBOARDED(113)
   The camera sends it when it is already onboarded and paired.
   In that case, you have no action to take from the user. You should show user that the camera is onboarded, close the connection with the camera and exit the process.

Sending Onboard and pairing Credentials to Camera

If you find that the camera needs to onboard or pair, you have to send the onboarding and pairing credentials to the camera. Basically camera needs onboarding credentials when the onboarding state indicates INDICATE_WAITING_FOR_ONBOARDING(111). It also expects onboarding credentials from the user if the onboarding state is INDICATE_DEVICE_ONBOARDED(113) and the connectivity state is INDICATE_CONNECTIVITY_DOWN(1505).

The Onboarding Credentials

To onboard a camera you need WiFi credential from the user. You can make and UI as per your need to take the WiFi credentials from user. Or use any other method according to your own business logic to collect a valid WiFi credentials as the onboarding credentials.

The Pairing Credentials

The pairing credential consists of

1. Pairing token
   Server gives this key to a user and when receives a pairing request from camera with a key, server finds to whom this key was given and pair the camera with that user account.

2. User Id
   Server uses this user id to find the pairing token given to this user. With this, the search space narrows down making the search process efficient for the server.

Getting The Pairing Token

📘

Android

Get your pairing token through Access.java with 3 simple steps.

1. First register a pairing token listener to accessthrough Access::RegisterPairingTokenListener() method.

Java

AnyConnectApi.get().getAccess().RegisterPairingTokenListener(pairingTokenListner);

2. Call Access::getPairingToken() method.

Java

AnyConnectApi.get().getAccess().getPairingToken();

3. You will receive the pairing token in the listener.
   see here about apiStatus

Java

pairingTokenListner = new Access.PairingTokenListener() {
    @Override
    public void onReceiveKey(int apiStatus, String pairingToken) {
        // you will receive `pairingToken` as second parameter
    }
}

📘

iOS

Get your pairing token through Access class with below simple steps.

1. Call getPairingToken(PairingTokenCallback callback) method of Access class to register callback function.

Declaration of PairingTokenCallback is:

Objective-c

typedef std::function<void(const int apiStatus, const PairingToken &pairingToken)> PairingTokenCallback;

2. You will receive the pairing token in the callback.
   see here about apiStatus

Objective-c

AccessRet getPairingTokenResult = access->getPairingToken([self](const int apiStatus, const PairingToken &pairingKey) {
        // apiStatus 0 for success. Error otherwise
        // See AccessTypes.h for more details of PairingToken
        });
}

Getting The User Id

📘

Android

Getting the user Id is very simple.
Just call Access::getUserId() method.

Java

String userId = AnyConnectApi.get().getAccess().getUserId();

📘

iOS

To get user Id call getUserId() method of AccessBuilder class.

Objective-c

NSString *userId = [NSString stringWithFormat:@"%llu",builder->getUserId()]];

📘

Android

To send onboard data to the camera, create an instance of CameraOnboardData by providing the pairingToken, userId, wifiSSID and wifiPassword. See the code snippet below.

Java

CameraOnboardData cameraOnboardData = CameraOnboardData.newBuilder()
                .withPairingKey(pairingToken)
                .withUserId(userId)
                .withWifiSsid(wifiSSID)
                .withWifiPassword(wifiPassword)
                .build();

Next, push the cameraOnboardData to CameraOnboardTask by invoking the pushOnboardData method of the CameraOnboardTask instance.

Java

cameraOnboardTask.pushOnboardData(cameraOnboardData);

The cameraOnboardTask starts to write the cameraOnboardData to a specific characteristic of the camera right after getting the data from the client. The write status of cameraOnboardData is provided to the onOnboardSdkEvent method of registered GattOnboardEventListener interface. The parameter of onOnboardSdkEvent is an enum which defines the write event of the onboard data. The OnboardSdkEvent can indicate any of the following events.

OnboardDataWriteStart : This event indicates that the onboard data write has just started.

OnboardDataWriteInProgress : This event indicates that the onboard data write is in progress.

OnboardDataWriteEnd : This event indicates that the onboard data write is successfully finished.

📘

iOS

Now combine these data to make a single NSString in the exact given format below.

Objective-c

NSString *onboarAndPairingdData = [[NSString alloc] initWithFormat:@"%@\n%@\n%@\n%@\n", wifiSsid, wifiPassword, userId, pairingToken];

Now convert the NSString to NSData and write to the peripheral.

Objective-c

NSData *dataToSend = [[NSData alloc] initWithData:[onboarAndPairingdData dataUsingEncoding:NSUTF8StringEncoding]];
[peripheral writeValue:dataToSend forCharacteristic:characteristic type:CBCharacteristicWriteWithResponse];

After receiving that data, camera will start to process the data and proceed for onboard and pair.
After receiving the data, it may take a while for camera to start sending next signals. So, it is advised to show user a progress during that period.

Processing the Received Signals from Camera

After receiving the onboard data, the camera will parse it.
The camera will start broadcasting the onboarding status through Notification state signal from now. Right after parsing the onboarding credentials, the notification state signal can be any of the following Data parsing signals.

1. INDICATE_PARSING_SUCCEEDED(201)
   Camera sends this signal when the data you have send is parsed successfully.

2. INDICATE_PARSING_FAILED(203)
   Camera sends this signal when it could not parsed the data you have send.
   In that case, you process will fail. You should let the user know about the failure, close the socket and exit.

When the data is parsed successfully, camera starts to connect to Internet by setting up the WiFi given to it.
At that time the camera will send WiFi setup signals

First, the camera will send

INDICATE_WIFI_SET_IN_PROGRESS(301)
It indicates the camera is setting the WiFi. You should show user a progress at this time.

After that one of the two signals you can see

1. If you send, invalid WiFi credentials, the camera will send a failure signal.
   INDICATE_WIFI_SET_FAILED(303)
   This means the camera has failed set up the WiFi received from application. At this point, you should notify user about the error, close the socket and exit.

2. In process of setting the WiFi, the camera will test if it can reach internet, So, it will send
   INDICATE_DEVICE_CONNECTIVITY_CHECK_IN_PROGRESS(121)
   The camera tries to reach the Internet while sending this signal. You should show user a progress at this time.

After the camera finish checking the connectivity to Internet, it will send

INDICATE_DEVICE_CONNECTIVITY_CHECK_DONE(123)
This is a state message. It doesn't indicate a success of failure.

At the next step camera lets you know the result of connectivity setup with Internet using WiFi setup signals.
It will send one of the two success and failure signal of WiFi setup signals.

INDICATE_WIFI_SET_SUCCEEDED(302)
Camera can reach the Internet successfully. You should show user a success message.

INDICATE_WIFI_SET_FAILED(303)
Camera has failed to reach the Internet. You should notify user about the error, close the socket and exit.

After setting up the WiFi, camera will try to pair itself with your account using the provided pairing token. At this point it will send the Pairing Signals.

First it will send

INDICATE_PAIRING_IN_PROGRESS(501)
Camera is trying to pair itself with the user account of which the pairing token and user id it has received. It this stage, you should show user a progress.

After the pairing process is done, it will show one of the two success of failure message.

INDICATE_PAIRING_SUCCEEDED(502)
This means the onboarding and pairing was successful. So, you should show a success message to user.

INDICATE_PAIRING_FAILED(503)
This means the onboarding and pairing process was unsuccessful. You should notify user about the failure, close the socket and exit.

If the camera onboarding and pairing is successful you will receive INDICATE_DEVICE_ONBOARDED(113) on both notification and onboard state signals at the same time.

You should show user that the camera is onboarded. Next, close the connection with the camera and exit the process if you are not collecting diagnostic reports from the camera. In case of collecting diagnostic reports, you should wait till getting INDICATE_SERVER_CLOSING(1307) from the camera. After getting this final signal, you should close the connection with camera and exit the onboard process.

To get the newly onboarded camera in your access control list, see this guide.

Collecting diagnostic report

For debug purposes, Smarter AI camera platform sends diagnostic informations of the onboarding process. This data is sent along with the state signal messages in the same characteristics of the camera through the GATT protocol.

📘

Android

Onboard log data is sent through the onReceiveOnboardLog method of the registered GattOnboardEventListener interface. This method is invoked on receiving each and every camera signal with a unique log data and the log data is passed as String through the parameter of onReceiveOnboardLog method. Concatenate the log data until the onboard session is finished.

Java

@Override
public void onReceiveOnboardLog(@NonNull String logData) {
    // logData contains the debug onboarding log of the camera
}

Closing the connection with the camera

When onboard process is completed, always close the bluetooth connection with the camera. Otherwise, the connection will continue to run in the background which will cause connectivity issues and unnecessary battery consumption of your device.

📘

Android

To close a bluetooth connection with the camera, invoke the closeCameraConnection() method of CameraOnboardTask.

Java

    cameraOnboardTask.closeCameraConnection();

📘

iOS

To close a bluetooth connection with the camera, invoke the cancelPeripheralConnection: method of CBCentralManager.

Objective-c

[self.centralManager cancelPeripheralConnection:peripheral];