Segmentation

The segmentation feature tells the software to fully cut out the visible people in the image before processing. Each cutout is called a "segment". Caption Pro can then use the face, text and number information within that segment to help identify the person.

For example, when reading a number on someone’s clothing, the software focuses on the information belonging to that person and ignores unrelated numbers on a scoreboard or on another athlete in the background.

Segmentation also helps exclude faces printed on posters, T-shirts and other clothing.

Open your project settings and enable Segmentation.

You can then choose how Caption Pro decides which people to process.

"On-demand processing" stops the software from performing face or number recognition on the image until a user specifies which segment they want to process. This is good for projects where you want to have full control of who gets processed, preventing any biometric information being used for background or unauthorised segments.

The drawback of this feature is that it is slower than standard, as a user needs to manually select each segment they want to process.

Enable Key person detection to let Caption Pro choose the main subjects in the image for you. It will then attempt to identify only those key people.

Segments it does not consider key subjects are left unprocessed. You can manually process those segments the same way as you would if on-demand processing was enabled.

For this feature, the software is instructed to try to identify the key action takers in the image, and it should be able to identify when an image is intended to include multiple individuals (a group posed up for instance) or if an image only has one key person running around a crowded environment.

With Key Person Detection enabled, the software will still cap the number of segments that get processed based on the number in the "Max Key People" field.

If both On-demand processing and Key person detection are disabled, Caption Pro processes only the largest segments, up to the limit set in Max key people.

For example, setting this to 10 tells the software to attempt to identify the people in the 10 largest segments.

These settings determine who Caption Pro attempts to identify. A person can be selected for processing without being successfully identified.

This feature is best if you simply want to ID everyone the software might know in the image.

When Text Recognition is enabled at the same time as Segmentation, the Text Recognition performance automatically gets a significant upgrade.

Caption Pro will only look within each segment (and combine all visible text information within that segment) to try to identify that person.

This means that background numbers are ignored and only those relevant ones (even multiple partial matches) are considered.

Additionally, it means that the software can consider things like jersey and kit colour when determining what player a certain number might belong to.

As such, with segmentation enabled, the auto-select feature is much more accurate, and you can trust the software to add matching names to the name-bar for you.

With Auto Select enabled, any matches that the software suggests will be automatically selected and added to the name bar for you (and subsequently will appear in all relevant formulas elsewhere in your metadata templates) - as long as the match is above your "auto select threshold".

For example, at 90%, a text recognition match must reach that score before the name is selected automatically.

Kit detection adds another check when matching people against your number lists: whether they are wearing the kit associated with that list.

With this feature enabled, if a number or text match is not wearing the described kit, they will not be auto-selected. As such, it is a useful feature if you are only wanting to ID names on jerseys from a specific team.

Alternatively, if you have multiple number lists active with, for example, a number "15" on both teams, it will know which person so select based on the jersey they are wearing.

To set it up:

For a well-known team, its name may be enough—for example, you can probably just write "San Francisco 49ers" in that field and the software will know who they are, regardless of what kit they are wearing.

For a less widely known team, or a limited edition kit, describe the kit’s appearance, such as yellow basketball jersey.

With Segmentation enabled, the software will tend to use more AI processes. In order to see a breakdown of AI processes for each image, check the "AI processes" indicator at the top of the UI:

As standard, this feature will have :
1 AI process per image

1 AI process to perform segmentation on the whole image
1 AI process for each face being processed in the image

1 AI process for each segment where Text Recognition is being used

This is in addition to the AI processes incurred with AI keywording or Auto-Captioning (1 per image per feature).

In some instances, fewer AI process are used because of background face filtering and due to the fact that the software only processes the key people, but because there is more processing power going into each segment's identification, on average our tests show that this feature tends to increase AI processes by roughly 20-25%.