Using the AI Vision Sensor in VEXcode V5 Python

This article provides an overview of the functions available to use the AI Vision Sensor in Python, along with simple instructions on how to use them. You will learn the four steps in the process of using the AI Vision Sensor – obtaining data, seeing the amount of objects obtained, selecting what object from the data to use, and what data from the object to pull.

Obtain Visual Data with the AI Vision Sensor

To gather information from its surroundings, the AI Vision Sensor takes an image of what it's currently looking at. Any objects that it identifies with the specified Visual Signature are put into a tuple, which can then be used by other functions.

take_snapshot

When using the take_snapshot command, you will need to create a variable to store the tuple that it creates. Any Visual Signatures will need to be configured before they can be used as a parameter. When using the function, specify the signature parameter as the sensor's name, double underscore, followed by the Visual Signature's name.utility exm.PNG

vision_objects = ai_vision_6.take_snapshot(ai_vision_6__Blue)
vision_objects = ai_vision_6.take_snapshot(ai_vision_6__Red)
vision_objects = ai_vision_6.take_snapshot(ai_vision_6__Green)

The options for your Visual Signature will also autofill in VEXcode after you start typing the beginning of the parentheses.

Example

In this example, the variable vision_objects stores a tuple containing the detected Visual Signatures from the AI Vision Sensor named ai_vision_6. It displays the number of objects detected and captures a new snapshot every 0.5 seconds.

while True: 

# Take a snapshot to create a tuple of detected objects
vision_objects = ai_vision_6.take_snapshot(ai_vision_6__Blue)

brain.screen.clear_screen()
brain.screen.set_cursor(1, 1)

brain.screen.print("object count:", len(vision_objects))
wait(0.5, SECONDS)

Determine How Many Objects were Detected

The len command can be used with tuples. This can be used to see how many objects the take_snapshot command has stored in the tuple.

It is best practice to always ensure you have detected an object before you try pulling any information from the snapshot. To do so, use the len command with the variable storing the tuple.

len(vision_objects) > 0

Example

In this example, the variable vision_objects stores a tuple containing the detected Visual Signatures from the AI Vision Sensor named ai_vision_6. It displays the number of objects detected and captures a new snapshot every 0.5 seconds. However, if it has detected an object, it will print how many objects it detected.

while True: 
# Take a snapshot to create a tuple of detected objects
vision_objects = ai_vision_6.take_snapshot(ai_vision_6__Blue)

# Clear the screen/reset so that we can display new information
brain.screen.clear_screen()
brain.screen.set_cursor(1, 1)

if len(vision_objects) > 0:
# Print how many objects were found
brain.screen.next_row()
brain.screen.print("object count:", len(vision_objects))
wait(0.5, SECONDS)

By using the len command with the tuple, you can also see how many objects were detected.

Select Which Object To Get Data From

There are two methods to extract data from the tuple. One approach is to directly retrieve information from the largest object detected in the most recent use of the take_snapshot command using the ai.vision.largest_object() function. The other method involves specifying the specific variable and object index.

ai.vision.largest_object()

ai.vision.largest_object() is a helper function to access the largest object in the tuple, but will only work with the most recent snapshot.

while True: 
# Take a snapshot to create a tuple of detected objects
vision_objects = ai_vision_6.take_snapshot(ai_vision_6__Blue)

# Clear the screen/reset so that we can display new information
brain.screen.clear_screen()
brain.screen.set_cursor(1, 1)

# Check if an object exists before pulling data
if len(vision_objects) > 0:
brain.screen.print(Center X: ", ai_vision.largest_object().centerX)
else:
brain.screen.print("no object")
wait(0.5, SECONDS)

In this example, the code checks if an object exists before using the ai.vision.largest_object() function to return the largest detected object's .centerX property and printing it.

vision_objects[0]

Another method to pull data from a tuple is by calling the variable storing the tuple directly. In brackets, specify the index of the object you want to extract data from. The returned tuple of objects always lists the largest object first at index 0. Subsequently, objects are ordered from largest to smallest.

while True: 
# Take a snapshot to create a tuple of detected objects
vision_objects = ai_vision_6.take_snapshot(ai_vision_6__Blue)

# Clear the screen/reset so that we can display new information
brain.screen.clear_screen()
brain.screen.set_cursor(1, 1)

# Check if an object exists before pulling data
if len(vision_objects) > 0:
brain.screen.print(Center X: ", vision_object[0].centerX)
else:
brain.screen.print("no object")
wait(0.5, SECONDS)

In this example, the code checks if an object exists before using the vision_object[0] function to return the 0 index object's .centerX property and printing it.

Select What Data to Pull From an Object

From each object, there are eight properties that can be called.

  • .height
  • .width
  • .centerX
  • .centerY
  • .originX
  • .originY
  • .angle
  • .exists

.height

The .height property returns how tall the detected object is in pixels.

Use this code to return the .height property:

vision_objects[0].height

The .height property will return a value between 0 and 240 pixels.

.width

The .width property returns how wide the detected object is in pixels.

Use this code to return the .width property:

vision_objects[0].width

The .width property will return a value between 0 and 320.

.centerX

The .centerX property reports the X coordinate of the center of the detected Visual Signature.

Use this code to return the .centerX property:

vision_objects[0].centerX

The .centerX property will return a number between 0 and 320.

.centerY

The .centerY property reports the Y coordinate of the center of the detected Visual Signature.

Use this code to return the .centerY property:

vision_objects[0].centerY

The .centerYproperty will return a number between 0 and 240.

.originX

The .originX property reports the X coordinate of the top-leftmost corner of the detected Visual Signature.

Use this code to return the .originX property:

vision_objects[0].originX

The .originXproperty will return a number between 0 and 320.

.originY

The .originY property reports the Y coordinate of the top-leftmost corner of the detected Visual Signature.

Use this code to return the .originY property:

vision_objects[0].originY

The .originYproperty will return a number between 0 and 240.

.angle

The .angle property returns the value of the Color Code's angle.

Use this code to return the .angle property:

vision_objects[0].angle

The .angle property returns the detected Color Code's orientation, ranging from 0 to 359.99 degrees.

.exists

.exists is a Boolean property that return true if it is a valid object. 

It will return false if the object is invalid, such as if a negative integer is used. If an object is invalid, all other properties will return 0s as their value.

while True: 
# Take a snapshot to create aa tuple of detected objects
vision_objects = ai_vision_6.take_snapshot(ai_vision_6__Blue)
# clear the screen/reset so that we can display new information
brain.screen.clear_screen()
brain.screen.set_cursor(1, 1)

if vision_objects[0].exists:
brain.screen.print("x:", vision_objects[0].centerX)
else:
brain.screen.print("no object")
wait(0.1, SECONDS)

In this example, the code checks the .exists property, if it returns true, it will print the .centerX property value to the Brain. If the .exists property returns false, it will print that there is no object.

For more information, help, and tips, check out the many resources at VEX Professional Development Plus

Last Updated: