Overview
FLOW Cortex extends standard FLOW Video Analytics with an additional AI-powered image analysis layer based on large language models (LLMs).
While FLOW Video Analytics handles real-time object detection, classification, tracking, and event generation, Cortex adds a flexible evaluation layer that can analyze image content using natural language prompts. Cortex processes images and returns responses to user-defined queries. This makes it possible to evaluate complex image attributes and situations that are not covered by standard FLOW Video Analytics.
Available since FLOW 2.3.0. The feature is automatically enabled and available to users.
Architecture
FLOW Cortex runs as a separate service that FLOW devices can connect to. TrafficCamera, TrafficXRoads, and TrafficEnterprise deployments can submit images to a configured Cortex provider and process the returned response.
The Cortex service is configured centrally and can be shared across multiple FLOW deployments. Individual devices decide when to send images for evaluation and how to use the responses. This design allows seamless integration without affecting core FLOW Video Analytics operation.
Figure: FLOW Cortex available on all FLOW devices.
Purpose and Typical Applications
FLOW Cortex is designed for image analysis tasks that are difficult or impractical to solve using conventional detectors and classifiers. Image processing with Cortex is computationally demanding. For this reason, Cortex is best suited for low-frequency tasks (generally not more frequent than once every 2–3 seconds).
Typical use cases include:
Cyclist vs. Pedestrian-with-bicycle classification – Distinguishing whether a person is riding a bicycle or walking with it. This affects traffic statistics and correct classification of traffic participants.
Passenger counting (person count in vehicles) – useful for occupancy monitoring, HOV lane verification, or transportation studies.
Seatbelt verification – checking whether visible occupants are wearing seatbelts.
Vehicle make and model recognition – returning structured information about manufacturer and model.
Emergency vehicle verification – detecting activated warning lights, visible emergency markings, or sirens.
Configuration
Cortex providers are configured in Interfaces → Cortex providers, where you can find Cortex Provider Definition, Live Chat testing functionality and Unit Status Details.
Figure: Cortex providers section under Interfaces.
In the overview you can also see the following columns:
Avg(s) – average response time of the provider
Last poll – time of last successful communication
Cortex status – current provider status.
The following status values may be displayed:
RUNNING – the provider is reachable and operating normally.
Reachable – the provider is reachable but has not reported the RUNNING state. If the provider returns a custom status (for example LOADING MODEL), it is displayed alongside the Reachable status.
Unreachable – communication with the provider failed.
Disabled – the provider is disabled in the configuration.
Unknown – the provider has not yet been contacted since startup.
Cortex Provider Definition
Here you define and manage connections to Cortex backend services. You can add multiple providers if needed.
Name
Address (URL where Cortex is running)
Bearer Token (provided)
Optional Key-Value pairs for additional Headers or Parameters for the request. Currently unused.
Figure: Cortex Provider definition details.
Cortex Live Chat
Cortex Live Chat is a built-in testing tool that enables you to quickly verify the connection to a Cortex provider and test various prompts and images.
Attach image(s) — Supports up to 8 images. You can upload them, paste directly from the clipboard using Ctrl+V, or use drag & drop. You can refer to them by their numbers later.
Send size — Allows you to adjust the resolution of the image sent to Cortex. Higher resolution provides more detail but increases processing load.
Prompt input — Enter your prompt in the text field, referring to the images attached. Note that the chat does not maintain conversation context — each submission is processed independently.
Figure: Cortex Live Chat feature.
If a request cannot be processed, Live Chat displays an error message.
Common causes include:
Invalid or expired authentication credentials.
Missing or invalid Cortex license.
Unauthorized model.
Insufficient credits.
Temporary Cortex server overload.
Invalid request.
Figure: Example of a Live Chat error message.
Cortex Unit Status Details
This option summarizes the status of the Cortex unit.
Status
Last poll
Requests (total from the Cortex unit)
Requests in system (local)
Avg processing time (s) – Response times consistently above 3-5 seconds indicate insufficient hardware performance, or too frequent or complex prompts.
System init (date and time)
Log entries (with direct access to logs)
Figure: Cortex unit status details.
A: Using Cortex with Image Evidence
Using Cortex with the Evidence module allows you to enrich generated evidence records with additional AI-generated information or to validate whether an event should be recorded at all.
Figure: Before usage, make sure that the Evidence feature is Enabled in the Analytics settings.
The configuration can be found in the Evidence → Settings. Here you can edit or create new Evidence tasks, you will see a settings subsection Cortex insights:
Figure: General Evidence Settings.
In the Cortex insights subsection, select the Provider and define:
Image query prompt
You may ask a question about the image. The Cortex response is added to the evidence payload in the Cortex section.
Validation prompt
You may ask a question to decide whether the evidence record should be created. The response is used to approve or reject event generation.
Additionally, in the Image evidence section you may affect which images collected for the evidence task are available to Cortex prompts. Assigned numbers may be used in references in prompts.
Figure: Cortex insights option in the Evidence task settings.
Cortex response for the query prompt is added to the details of each Evidence record under their dedicated Cortex section.
Figure: Evidence record details with the Cortex section.
B: Using Cortex Watcher for Continuous Evaluation
Cortex Watcher provides continuous, periodic evaluation of selected areas of the camera view. Unlike Image Evidence, which is event-triggered, Cortex Watcher runs automatically, respecting a defined time period, and displays results directly on the Dashboard.
Watchers are configured in Analytics settings → Cortex watcher settings.
Define regions of interest (regions are automatically converted to rectangles).
For each region: assign a Cortex provider, prompt, and evaluation period.
You can reduce the computational load by adjusting the region’s resolution. (Resolution set bigger than the original region is automatically fixed).
Figure: Cortex Watcher feature.
Outputs are displayed on the Dashboard as widgets showing the Cortex response string. Disabled watchers return the default output: "FLOW: Disabled".
Figure: Cortex watcher output on the Dashboard. and a Disabled watcher’s output. Note that the Cortex response in this example is complex - avoid generic prompts such as "Analyze image" and prefer stricter prompts.
Creating Effective Prompts – Best Practices
Prompt design has a significant impact on response quality,processing performance, and consistency. Consider the following recommendations when creating Cortex prompts:
DO provide clear instructions and define the expected output structure (JSON, yes/no, short enumeration, etc.).
DO limit the range of possible answers and specify the exact response format.
AVOID open-ended requests that generate long textual responses, as these are the most computationally expensive to process.
Example: Vehicle Analysis Prompt
The following example shows a structured vehicle analysis prompt returning results in a predefined format.
You will receive ONE image containing a vehicle. Return ONLY plain text in the following format:
"manufacturer": "<string or null>",
"model": "<string or null>",
"license_plate": "<string or null>",
"headlights": <true|false>,
"number_of_passengers": <integer>
Rules:
- manufacturer: vehicle brand (e.g. Volkswagen, Toyota, BMW), or null if uncertain.
- model: vehicle model, or null if uncertain.
- license_plate: exact visible plate text, or null if not readable.
- headlights: true if clearly on, otherwise false.
- number_of_passengers: visible occupants including the driver.
Additional instructions:
- If multiple vehicles are present, analyze the largest vehicle in the foreground.
- Do not add explanations, comments, or additional fields.
- Keep the exact key order shown above.