About AI Data Capture SDK

AI Data Capture SDK 4.0

Overview

The Zebra AI Data Capture SDK enables Java and Kotlin developers to create computer vision applications on Zebra mobile computers, offering tools and resources for both experienced and novice AI professionals to leverage the AI capabilities of Zebra devices.

As part of Zebra’s Frontline AI Enablers, which includes various AI Models tailored for enterprise use cases, the SDK offers APIs to use these models. Developers can pass frames or images from the camera or other source to the models using traditional methods or they can accelerate development with CameraX Analyzers. One such offering is the EntityTrackerAnalyzer, which enables developers to quickly and easily build CameraX-based applications for tracking Entities such as barcodes or text.

The AI Models include:

  • Barcode Decoder Model - Detects and decodes 1D and 2D barcodes within an image.
  • Product and Shelf Recognizer Model - Detects and recognizes products displayed on retail shelves from captured images. This model can also detect additional elements such as shelf labels, peg labels, and shelves themselves.
  • TextOCR Model - Detects and recognizes text within an image.

While the AI Data Capture SDK provides built-in detectors for common enterprise use cases — such as BarcodeDecoder, Localizer, and TextOCR — specialized applications often require custom models or external SDKs. To bridge this gap, the SDK introduces CustomDetector, empowering developers to seamlessly integrate any third-party machine learning model directly into the existing tracking pipeline alongside Zebra's native detectors.

Blueprints are ready-made, adaptable AI frameworks, built on Zebra's edge-optimized models, that transform concepts into practical solutions for modernizing frontline work by automating tasks, reducing errors, and simplifying the adoption of AI into unique workflows.

The AI Data Capture SDK also offers a built-in ViewFinder (EntityViewfinder), capable of processing and rendering the interactive Entities generated during the session.

Key Benefits:

  • Optimized for Peak Performance on Zebra Devices - The Frontline AI Enabler Models and the AI Data Capture SDK are specifically fine-tuned to harness the full potential of Zebra’s mobile computers, ensuring maximum efficiency and performance.
  • Effortless Integration Across Devices - Developers can leverage trained Frontline AI Enabler Models on Zebra devices, simplifying the integration of AI vision into enterprise applications without the need for custom model training.
  • Simplified Development Workflow - The AI Data Capture SDK integrates effortlessly with the CameraX framework, streamlining the development of vision-based applications and minimizing development complexity. For more information, refer to EntityTrackerAnalyzer and EntityViewfinder.

Overview Video - This video provides an overview of Zebra's Frontline AI Enablers:

SDK Capabilities

SDK capabilities:

Barcode Localizer
Product and Shelf Localizer
Product Recognition
Barcode Decoder
image image image image
Automatically detects all barcodes within an image, eliminating the need for individual scans. An augmented reality overlay highlights barcodes on labels, boxes, and shelf tags for decoding. Identifies products on shelves, enabling tasks such as locating a specific product, determining restock needs, and processing Point of Sale (POS) sales. Adapts to various aisle widths with flexible reading distances. Builds and stores detailed visual data about retail products, enhancing inventory management. This process typically follows the use of Shelf Localizer, which detects products in images, crops them and submits them for recognition. Detects and decodes various types of barcodes within an entire image or from specific regions.
Text OCR
Entity TrackerAnalyzer
Entity Viewfinder
image image image
Detects and recognizes text and characters in an image, converting it into words with high accuracy. Supports various fonts, font sizes, orientations, and lighting conditions. Detects, decodes, recognizes and tracks Entities such as barcodes, text, shelves or products in real-time using images or video, with built-in tracking that assigns persistent IDs for linking actions to entities. A built-in viewfinder designed to deliver a customizable and interactive camera viewfinder interface.
Image Attributes Detector
Image Transform Detector
Custom Detector
image
Analyzes images in real-time to detect and verify their content and quality, detecting critical attributes like blur, object presence, and scene characteristics. Detects and blurs sensitive information within images, including barcodes, text, and people. Streamlines the integration of third-party machine learning models and SDKs directly into the AI Data Capture SDK pipeline, extending processing capabilities beyond native Zebra models.

Each of these capabilities can be used individually or combined to streamline tasks across various industries.

Barcode Localizer

Barcode Localizer detects 1D and 2D barcodes in images, suitable for various use cases such as identifying barcodes on product boxes, shelves and shipping labels.

Product and Shelf Localizer

Product and Shelf Localizer detects and identifies objects on retail shelves, aiding inventory management, optimizing space and ensuring accurate labeling. The types of objects detected include:

  • Products - Identifies individual products on the shelf, facilitating inventory tracking and automating stock checks.
  • Shelf Labels - Detects and reads shelf labels, ensuring that products are accurately priced and labeled.
  • Peg Labels - Recognizes peg labels used for hanging products, aiding in efficient product organization.
  • Shelves - Detects the presence and structure of shelves themselves, helping in understanding shelf layouts and optimizing space usage.

These localizers are usually followed by a decoding and recognition phase:

  • Barcode Localizer - The image, along with its localized bounding boxes, can be passed on to the BarcodeDecoder to decode barcode data. Both detection and decoding can be performed simultaneously with the use of the process() API.
  • Product and Shelf Localizer - The bounding boxes identified by the localizer can be used to recognize the products.

Product Recognition

Product Recognition builds a database of stored products (product enrollment), enabling their recognition for use cases such as inventory tracking and price compliance. The Feature Extractor isolates key features from images, generating descriptors - vectors of float values that capture an item's characteristics - and stores them in Feature Storage to enable product recognition. After a database of recognizable products is established, the Product Recognizer performs semantic searches to locate matching descriptors, predicting the identities of products on the shelf.

Barcode Decoder

The Barcode Decoder detects and decodes various types of barcodes in images. It first identifies the location of barcodes within captured images, and then decodes them from either the entire image or from specific regions.

Text OCR

The Text OCR model detects and decodes text characters in images, offering suggestions for recognized characters or words. It adapts to various fonts and input sizes, allowing for effective text recognition at different distances. Detected words can be grouped into 'lines' or 'paragraphs.'

EntityTrackerAnalyzer

EntityTrackerAnalyzer is a CameraX-compatible implementation of the ImageAnalysis.Analyzer interface, designed for real-time detection, decoding, recognition, and tracking of Entities using still images or video frames. An Entity represents any element detectable by the AI Data Capture SDK, such as a barcodes, text, shelves or products, enabling various user applications. The analyzer includes built-in tracking capabilities, assigning a unique track ID to each Entity that persists as long as the Entity remains within view, allowing developers to link visual or operational actions to the tracked entities. Seamlessly integrating with CameraX, the EntityTrackerAnalyzer processes image frames using a series of detectors to deliver aggregated Entity tracking results, efficiently handling asynchronous tasks and lifecycle events for smooth operation within applications.

Note: Currently, EntityTrackerAnalyzer is designed to detect, decode, and track barcodes.

EntityViewfinder

The EntityViewfinder is a built-in, customizable viewfinder that serves as a CameraX preview, with the ability to render Entities. It offers an enhanced user experience compared to the default PreviewView. It offers seamless integration into XML-based UI layouts and configuration of attributes such as zoom levels, flash states, and button visibility. It also supports features like drag-and-drop repositioning and enforces minimum size constraints for optimal usability. It consists of two key components:

  • EntityView - Responsible for rendering the visual UI, including essential controls such as zoom, flash, and resizing.
  • EntityViewController - Manages operations such as camera preview, entity rendering, and user interactions.

For advanced customization, developers can use StylePen implementations to render bounding boxes or icons around detected entities. This makes the EntityViewfinder a versatile tool for applications requiring real-time visual feedback and interaction.

image
Components of AI Data Capture SDK

Image Attributes Detector

The Image Attributes Detector class offers a simple, asynchronous method for analyzing images in real-time to verify their content and quality. It detects critical attributes like blur, object presence, and scene characteristics, to instantly validate that an image is clear and contains the correct subject matter. This makes it ideal for enterprise applications in logistics and proof-of-delivery, and other workflows benefit where image integrity is critical.

Image Transform Detector

The Image Transform Detector class transforms images by detecting and blurring sensitive information, like barcodes, text, and people. This asynchronous, real-time operation can be combined with the Image Attributes Detector to create a single, streamlined workflow for both image attributes analysis and transformation.

Custom Detector

The Custom Detector class offers a streamlined method for integrating third-party machine learning models and SDKs directly into the AI Data Capture SDK pipeline, extending processing capabilities beyond native Zebra models. This significantly expands the AI Data Capture SDK, allowing the seamless integration and processing of various external model frameworks.

Blueprints

Blueprints transform AI concepts into practical, real-world results. Each Blueprint is a ready-made, adaptable framework that demonstrates AI modernizing high-volume, manual frontline work. They provide a proven starting point for automating repetitive tasks, reducing errors, and scaling AI solutions. Built on Zebra’s edge-optimized AI models, Blueprints make it easy to see, test, and adapt AI capabilities to unique workflows.

Picture Proof of Delivery

Picture Proof of Delivery (PPOD) is an on-device AI Blueprint that guides drivers to capture a compliant photo on the first try. It addresses the common challenges of time pressure, complex image requirements (showing the parcel and surroundings), and the need to avoid capturing sensitive data (like people or license plates).

Two distinct workflows are offered:

  • Guided PPOD - Provides real-time user guidance to ensure a compliant photo is captured on the first attempt.
  • Non-Guided PPOD - Enables flexible and efficient post-capture and redaction for images that have already been taken.

Getting Started

Get Started with Zebra's Frontline AI Enablers

Follow these steps to build applications with the AI Data Capture SDK:

  1. Download the required model for the component you plan to use:
  2. Download the AI Data Capture SDK to integrate the functionality into your Android project.
  3. Access the developer resources for implementation guidance and API references:
  4. Explore Demo Apps - Test real-world scenarios using the Zebra Showcase App. For installation instructions, click here. The source code is available to help developers build production-ready applications faster:
     Computer man            Computer man
AI Barcode Finder Demo       AI Data Capture Demo

Additional Developer Resource

For comprehensive guidance on building apps with this SDK, visit Zebra's A Practical Guide to the Zebra Frontline AI Enablers.

New in v4.0

New in v4.0.6

  • Updated and enhanced all models for compatibility with the latest SDK version: All models now require AI Data Capture SDK v4.0 or higher.
  • New Beta Feature to support model-based barcode decoding via enableAIBarcodeDecoding value from BarcodeDecoder.
  • Updated default values for Image Attributes in the Proof of Delivery Model to enhance performance. To review the previous default values, refer to the Image Attributes Proof of Delivery Model section.
  • New Custom Detector that seamlessly integrates third-party machine learning models and SDKs into the AI Data Capture SDK, expanding application processing capabilities beyond Zebra's default models.
  • Added support for TC501 and TC701 (Q-6690-platform devices); see Device Requirements.

Version History

New in v3.3

  • Includes bug fixes to improve barcode decoding performance.

New in v3.2

New in v3.2.3

  • Updated the AI model for Image Attributes to version 2.1.0, resulting in more precise attribute detection and new default values.
  • The Pallet and Box Localizer Model (Beta) (formerly Warehouse Localizer) detects boxes, pallets, warehouse shelves, and shelf labels from RGB images.
  • Resolved Issues:
    • The getCorners() method in BarcodeEntity now correctly returns the barcode's corner points.
    • Resolved an issue where the Module Recognizer’s process(ImageData imageData) API now throws a specific exception for invalid input, enabling more precise error handling.

New in v3.2.8

  • New Picture Proof of Delivery (PPOD) feature ensures delivery photos are compliant and have sensitive information automatically redacted, offering two distinct workflows:
    • Guided PPOD - Provides real-time user guidance to ensure a compliant photo is captured on the first attempt.
    • Non-Guided PPOD - Enables flexible and efficient post-capture and redaction for images that have already been taken.
  • New Methods:
    • LabelEntity:
    • getBarcodes() - Retrieves barcode data from the label.
    • ShelfEntity:
    • getLabels() - Retrieves a list of labels associated with the shelf.
    • getProducts() - Retrieves a list of products associated with the shelf.
  • ModuleRecognizer Enhancements:
    • Barcode Recognition - The recognizer can now detect and decode barcodes found on shelf labels.
    • New enableBarcodeRecognition() Method - Enables the recognition of products identified by matching detected products against entries in the specified index and label files.
  • The SKUInfo constructor has been updated to accept a third parameter, normalizedAccuracy, in addition to the existing productSKU and accuracy parameters. This normalized accuracy score can be retrieved using getNormalizedAccuracy().

New in v3.1

  • Added Text OCR detector support for EntityTrackerAnalyzer.
  • New DisableAllSymbologies() method allows developers to disable decoding for all barcode symbologies, providing the flexibility to selectively enable only the required ones.
  • Updated several entity classes, including the addition and removal of multiple methods in BarcodeEntity, LocalizerEntity, LineEntity, ParagraphEntity, and WordEntity.
  • Deprecated and removed the DecodedTextEntity class.

New in v3.1.6

  • Improved stability and performance of the SDK through various fixes.
  • Implemented 16kb page alignment to prevent an SDK warning.

New in v3.1.7

  • Added ModuleRecognizer detector support for EntityTrackerAnalyzer. The new ModuleRecognizer simplifies product recognition by providing a unified, end-to-end pipeline for detection and recognition.
  • Introduced new Entity classes:
    • LabelEntity - Detect and classify labels within a retail shelf image, identifying the specific type (shelf label or peg label) and the coordinates of each detected label.
    • ProductEntity - Identify and locate products within a shelf image, enabling integration of product detection and recognition into retail automation workflows.
    • ShelfEntity - Identify and locate shelf regions within an image, providing spatial content for analyzing product and label placement.
  • Added support for Q-6690-platform devices, including ET401.
  • Released new models for:

New in v3.1.10