Skip to content

Use with AI

Use with AI

Optonic Mikado Find ​

Mikado Find is a complete package of camera and software for recognizing 3D objects.

This article is valid for all horstOS versions as well as for horstFX > 2021.07.

1. Introduction ​

The point cloud from any Ensenso 3D camera is scanned against the CAD model. Position, orientation, and matching score of the identified parts are provided through the integrated OPC-UA interface. The intuitive calibration process allows for quick and easy setup at the facility.

2. Versions ​

This article is applicable for Mikado Find version > 1.1

Robot control software version > 2021.07

Python version > 3.73

3. Setting up the System ​

For videos and documentation on setting up the system, please visit the following link: https://www.optonic.com/en/products/mikado/find/

4. Communication ​

Mikado Find communicates through an OPC-UA interface using a Python OPC-UA Client. The installation and operation details are further explained in the following documentation. OPC UA. To test the communication, it is recommended to download the OPC-UA GUI from https://github.com/FreeOpcUa/opcua-client-gui.

The Python script can be executed directly on the control cabinet. Details on how it is automatically called via horstOS are described in the section on Executing External Python Scripts.

5. Example Program ​

The following program serves as a comprehensive programming example. The Python script for communication via OPC UA is directly called from the horstOS program. The relevant commands for connecting to the camera, loading the camera configuration, and sending the trigger command are sent from horstOS to the Python script using TCP IP Socket. Subsequently, the Python script forwards this information to the camera via OPC UA. The object coordinates of the detected objects are then transmitted from the camera to the Python script, which sends this data back to horstOS using TCP IP.

In horstOS, a workspace control then takes place. This involves checking whether the robot is within a desired range when approaching a gripping point and not colliding, for example, with the cell wall. If the workspace is valid, the object is grasped and placed in a pallet. Once the pallet is full, the objects are randomly thrown back into the box for an endless loop.

5.1. Python Script ​

The Python script is designed in a way that typically does not require modification for other program sequences. The focus should be on ensuring synchronization between horstOS and this script.

#!/usr/bin/python3

import sys
import time
import socket
sys.path.insert(0, "..")

from opcua import Client
from opcua import ua
from numpy import uint32

# definition of function to execute commands via OPC UA
def executeCommand(cmd):
    while outBusyBitNode.get_value():                               # wait for busy bit = false
        time.sleep(0.001)

    inCommandNoNode.set_value(cmd, ua.VariantType.Int32)            # set Trigger 

    while not outBusyBitNode.get_value():                           # wait for busy bit = true
        time.sleep(0.001)
    inCommandNoNode.set_value(0, ua.VariantType.Int32)              # reset trigger
                    
    while outBusyBitNode.get_value():                               # wait for busy bit = false
        time.sleep(0.001)

        
if __name__ == "__main__":

    # Connection to Camera via OPC-UA
    client = Client("opc.tcp://192.168.0.50:4840")      # connect using an IP Adress

    # Socket Connection
    HOST = "0.0.0.0"                                    # IP HORST 
    PORT = 3000                                         # Port HORST 

    try:        
        # connect to camera via OPC-UA
        print("connect to camera")
        client.connect()

        # connect to horstOS/FX va Socket
        s = socket.socket()                             # Create a socket object.
        s.bind((HOST, PORT))                            # Bind the port.
        print("socket listen for client")
        s.listen(5)                                     # Awaiting client connection.
        c, addr = s.accept()                            # Establish a connection with the client.
        print("socket accepted by client")
         
        doStop = False

        # get Objects node, this is where we should put our nodes
        root = client.get_root_node()

        # populating address space
        outBusyBitNode = root.get_child(["0:Objects", "2:Outputs", "2:BusyBit"])
        outResultNoNode = root.get_child(["0:Objects", "2:Outputs", "2:ResultNo"])
        outResultsNode = root.get_child(["0:Objects", "2:Outputs", "2:Results.50"])
        outErrorNoNode = root.get_child(["0:Objects", "2:Outputs", "2:ErrorNo"])

        inCameraNameNode = root.get_child(["0:Objects", "2:Inputs", "2:CameraName"])
        inRotationModeNode = root.get_child(["0:Objects", "2:Inputs", "2:RotationMode"])
        inCommandNoNode = root.get_child(["0:Objects", "2:Inputs", "2:CommandNo"])
        inConfigNameNode = root.get_child(["0:Objects", "2:Inputs", "2:ConfigName"])
        inSTLNameNode = root.get_child(["0:Objects", "2:Inputs", "2:STLName"])


        # open camera
        print("open camera")
        cameraName = "204929"                           # name of the camera
        inCameraNameNode.set_value(cameraName)          # set camaera name 
        executeCommand(6)                               # call execute function to execute the command

        if outErrorNoNode.get_value() == 0:
            print("connected to camera")
            c.sendall("Connection to the camera established successfully.".encode('utf-8'))

        else :
            print("connection to camera could not be established")
            c.sendall("Connection to the camera could not be established.".encode('utf-8'))
                        
        # When is the camera fully opened? Is there a command for this from the camera?
        #time.sleep(8)
        
        # load camera configuration
        configName = c.recv(1024).decode()              # Reads a message from HORST via a socket connection.
        print("load configuration:", configName);
        inConfigNameNode.set_value(configName)          # set config file name
        executeCommand(1)                               # call execute function to execute the command

        if outErrorNoNode.get_value() == 0:
            print("configuration has been loaded")
            c.sendall("Configuration successfully loaded".encode('utf-8'))

        else :
            print("configuration could not be loaded")
            c.sendall("Error loading the configuration occurred.".encode('utf-8'))

        #time.sleep(5)
        
        inRotationModeNode.set_value("sxyz")            # set Rotation Mode
        
        while (not doStop):

            try:
                msg = c.recv(1024)                      # Reads a message from HORST via a socket connection.

#************************************************************************************************trigger camera*******************************************************************************************
                
                if msg == b'Trigger':                   # check message of HORST

                    executeCommand(4)                   # call execute function to execute the command


#******************************************************************************'***************get camera results****************************************************************************************'
                        
                    # print("Error Number: ", outErrorNoNode.get_value())
                    
                    if outErrorNoNode.get_value() == 0:                                         # check camera for errors

                        print("Result Number: ", outResultNoNode.get_value())               

                        if outResultNoNode.get_value() != 0:                                    # check if camera has detected objects
                      
                            #cam_values_str = str(outResultsNode.get_value())
                            
                            cam_values = outResultsNode.get_value()
                            
                            # sort the detected objects regarding the object score
                            cam_values.sort(key=lambda x: float(x[0]), reverse=True)
                            print("Camera Object Values:\n", cam_values)

                            cam_values_str = str(cam_values)
                            c.sendall(cam_values_str.encode('utf-8'))                           # send complete camera value message
                            
                            #c.sendall(str(outResultsNode.get_value()[0][0]).encode('utf-8'))   # send just one camera value

                        else:
                            c.sendall("Kein Objekt gefunden".encode('utf-8'))
                    else :
                        c.sendall("Fehler".encode('utf-8'))

                        if outErrorNoNode.get_value() == 1:
                            ErrorMsg = "Unknown system error"

                        elif outErrorNoNode.get_value() == 2:
                            ErrorMsg = "Unable to open the camera."
                            
                        elif outErrorNoNode.get_value() == 3:
                            ErrorMsg = "No stereo camera connected"

                        elif outErrorNoNode.get_value() == 4:
                            ErrorMsg = "Camera Error"

                        elif outErrorNoNode.get_value() == 5:
                            ErrorMsg = "Search failed"

                        elif outErrorNoNode.get_value() == 6:
                            ErrorMsg = "Search did not find a corresponding model."
                            
                        elif outErrorNoNode.get_value() == 7:
                            ErrorMsg = "Model generation failed."

                        elif outErrorNoNode.get_value() == 8:
                            ErrorMsg = "Setting the 3D reference point failed."

                        elif outErrorNoNode.get_value() == 9:
                            ErrorMsg = "Configuration could not be loaded"

                        elif outErrorNoNode.get_value() == 10:
                            ErrorMsg = "Configuration could not be saved"
                            
                        elif outErrorNoNode.get_value() == 11:
                            ErrorMsg = "Command not implemented"

                        elif outErrorNoNode.get_value() == 12:
                            ErrorMsg = "Invalid index for retrieving the results"

                        elif outErrorNoNode.get_value() == 13:
                            ErrorMsg = "STL file not found"

                        c.sendall(ErrorMsg.encode('utf-8'))
                          
                else:
                    print("Program interrupted by HORST")
                    doStop = True

            except KeyboardInterrupt:
                print("Interrupted!")
                doStop = True
        
    finally:
        client.disconnect()
        c.close()

5.2. horstOS Script ​

From the horstOS example, some of the functions can be directly utilized for various applications. In some cases, simply adjusting the waypoints may suffice.

6. Downloads ​

horstOS program Mikado Find

OPC UA Client Mikad Find