NOTE: It is a preview version of the Inference Engine Python* API for evaluation purpose only. Module structure and API itself may be changed in future releases.
This API provides a simplified interface for Inference Engine functionality that allows to:
Currently the Inference Engine Python* API is supported on Ubuntu* 16.04 and 18.04, Windows* 10, macOS* 10.x and CentOS* 7.3 OSes. Supported Python* versions:
| Operating System | Supported Python* versions: |
|---|---|
| Ubuntu* 16.04 | 2.7, 3.5, 3.6, 3.7 |
| Ubuntu* 18.04 | 2.7, 3.5, 3.6, 3.7 |
| Windows* 10 | 3.5, 3.6, 3.7 |
| CentOS* 7.3 | 3.4, 3.5, 3.6, 3.7 |
| macOS* 10.x | 3.5, 3.6, 3.7 |
To configure the environment for the Inference Engine Python* API, run:
source <INSTALL_DIR>/bin/setupvars.sh .call <INSTALL_DIR>\deployment_tools\inference_engine\python_api\setenv.batThe script automatically detects latest installed Python* version and configures required environment if the version is supported. If you want to use certain version of Python*, set the environment variable PYTHONPATH=<INSTALL_DIR>/deployment_tools/inference_engine/python_api/<desired_python_version> after running the environment configuration script.
This class represents an Inference Engine entity and allows you to manipulate with plugins using unified interfaces.
__init__(xml_config_file: str = "")
xml_config_file - A full path to .xml file containing plugins configuration. If the parameter is not specified, the default configuration is handled automatically.IECore object with default configuration: IECore object with a custom configuration location specified: .xml file has the following structure: available_devices - A vector of devices. The devices are returned as [CPU, FPGA.0, FPGA.1, MYRIAD]. If there are more than one device of a specific type, they all are listed followed by a dot and a number.get_versions(device_name: str)namedtuple object with versions of the plugin specifieddevice_name - Name of the the registered pluginVersions namedtuple object with the following fields:major - major plugin integer versionminor - minor plugin integer versionbuild_number - plugin build number stringdescription - plugin description stringload_network(network: IENetwork, device_name: str, config=None, num_requests: int=1)ExecutableNetwork object of the IENetwork class. You can create as many networks as you need and use them simultaneously (up to the limitation of the hardware resources).network - A valid IENetwork instancedevice_name - A device name of a target pluginnum_requests - A positive integer value of infer requests to be created. Number of infer requests is limited by device capabilities.config - A dictionary of plugin configuration keys and their valuesExecutableNetwork objectquery_network(network: IENetwork, device_name: str, config=None)network - A valid IENetwork instancedevice_name - A device name of a target pluginconfig - A dictionary of plugin configuration keys and their valuesset_config(config: dict, device_name: str)config - a dictionary of configuration parameters as keys and their valuesdevice_name - a device name of a target pluginset_affinity method of the IENetwork class.register_plugin(plugin_name: str, device_name: str = "")plugin_name - A name of a plugin. Depending on a platform, plugin_name is wrapped with a shared library suffix and a prefix to identify a full name of the librarydevice_name - A target device name for the plugin. If not specified, the method registers a plugin with the default name.register_plugins(xml_config_file: str).xml configuration filexml_config_file - A full path to .xml file containing plugins configurationunregister_plugin(device_name: str = "")device_name - A device name of the plugin to unregisteradd_extension(extension_path: str, device_name: str)extension_path - Path to the extensions library file to load to a plugindevice_name - A device name of a plugin to load the extensions toget_metric(device_name: str, metric_name: str)ExecutableNetwork agnostic, such as device name, temperature, and other devices-specific values.get_config(device_name: str, metric_name: str)This class stores main information about the layer and allow to modify some layer parameters
name - Name of the layertype- Layer typeprecision - Layer base operating precision. Provides getter and setter interfaces.layout - Returns the layout of shape of the layershape - Return the list of the shape of the layerparents - Returns a list, which contains names of layers preceding this layerchildren - Returns a list, which contains names of layers following this layeraffinity - Layer affinity set by user or a default affinity set by the IEPlugin.set_initial_affinity() method. The affinity attribute provides getter and setter interfaces, so the layer affinity can be modified directly. For example:
To correctly set affinity for the network, you must first initialize and properly configure the HETERO plugin. set_config({"TARGET_FALLBACK": "HETERO:FPGA,GPU"}) function configures the plugin fallback devices and their order. plugin.set_initial_affinity(net) function sets affinity parameter of model layers according to its support on specified devices.
After default affinity is set by the plugin, override the default values by setting affinity manually how it's described in example above
To understand how default and non-default affinities are set:
net.layers function right after model loading and check that layer affinity parameter is empty.plugin.set_default_affinity(net).net.layers and check layer affinity parameters to see how plugin set a default affinitynet.layers again and check layer affinity parameters to see how it was changed after manual affinity settingweights- Dictionary with layer weights, biases or custom blobs if anyparams - Layer specific parameters. Provides getter and setter interfaces to get and modify layer parameters. Please note that some modifications can be ignored and overwriten by target plugin (e.g. modification of convolution kernel size will be reflected in layer parameters but finally the plugin will ignore it and will use initial kernel size)This class contains the information about the network model read from IR and allows you to manipulate with some model parameters such as layers affinity and output layers.
__init__(model: [bytes, str], weights: [bytes, str], init_from_buffer: bool=False, ngrpah_compatibility: bool=False)
model - An .xml file of the IR. Depending on init_from_buffer value, can be a string path or bytes with file content.weights - A .bin file of the IR. Depending on init_from_buffer value, can be a string path or bytes with file content.init_from_buffer - Defines the way of how model and weights attributes are interpreted. If True, attributes are interpreted as strings with paths to .xml and .bin files of IR. If False, they are interpreted as Python bytes object with .xml and .bin files content.IENetwork object from IR files: IENetwork object bytes with content of IR files: name - Name of the loaded networkinputs - A dictionary that maps input layer names to InputInfo objects. For example, to get a shape of the input layer: outputs - A dictionary that maps output layer names to OutputInfo objects For example, to get a shape of the output layer: batch_size - Batch size of the network. Provides getter and setter interfaces to get and modify the network batch size. For example: layers - Return dictionary that maps network layer names to IENetLayer objects containing layer properties in topological order. For example, to list all network layers: stats - Returns LayersStatsMap object containing dictionary that maps network layer names to calibration statistics represented by LayerStats objects. LayersStatsMap class inherited from built-in python dict and overrides default update()method to allow to set or modify layers calibration statistics. from_ir(model: str, weights: str) NOTE: The function is deprecated. Please use the
IENetwork()class constructor to create valid instance ofIENetwork.
.xml and .bin files of the IR..xml file of the IR.bin file of the IRIENetwork classadd_outputs(outputs)outputs - List of layer to be set as model outputs. The list can contain strings with layer names to be set as outputs or tuples with layer name as first element and output port id as second element. In case of setting one layer as output, string or tuple with one layer can be provided.NOTE: The last layers (nodes without successors in graph representation of the model) are set as output by default. In the case above,
problayer is a default output andconv5_1/dwise,conv2_1/expandare user-defined outputs.
reshape(input_shapes: dict)NOTE: Before using this method, make sure that the target shape is applicable for the network. Changing the network shape to an arbitrary value may lead to unpredictable behaviour.
input_shapes - A dictionary that maps input layer names to tuples with the target shapeserialize(path_to_xml, path_to_bin)path_to_xml - Path to a file, where a serialized model will be storedpath_to_bin - Path to a file, where serialized weights will be storedLayer calibration statistic container.
__init__(min: tuple = (), max: tuple = ())min - Tuple with per-channel minimum layer activation valuesmax - Tuple with per-channel maximum layer activation valuesThis class contains the information about the network input layers
precision - Precision of the input data provided by user. Provides setter and getter interfaces to get and modify input layer precision. List of applicable precisions: FP32 FP16, I32, I16, I8, U32, U16 NOTE: Support of any calculation precision depends on the target plugin.
layout - Layout of the input data provided by user. Provides setter and getter interfaces to get and modify input layer layout. List of applicable layouts: NCHW, NHWC, OIHW, C, CHW, HW, NC, CN, BLOCKEDshape - input layer data shapeThis class contains the information about the network input layers
precision - Precision of the output data. Provides setter and getter interfaces to get and modify output layer precision.layout - Layout of the output data provided by usershape - Input layer data shapeThis class is the main plugin interface and serves to initialize and configure the plugin.
__init__(device: str, plugin_dirs=None)device - Target device name. Supported devices: CPU, GPU, FPGA, MYRIAD, HETEROplugin_dirs - List of paths to plugin directoriesdevice - a name of the device that was specified to initialize IEPluginversion - a version of the pluginload(network: IENetwork, num_requests: int=1, config=None)network - A valid IENetwork instancenum_requests - A positive integer value of infer requests to be created. Number of infer requests may be limited by device capabilities.config - A dictionary of plugin configuration keys and their valuesset_initial_affinity(net: IENetwork)IEPlugin was initialized for a HETERO device.net - A valid instance of IENetworkaffinity attribute of the IENetLayer class.add_cpu_extension(extension_path: str)extension_path - A full path to CPU extensions libraryReturn value: None
set_config(config: dict)
SetConfig() in Inference Engine C++ documentation for acceptable keys and values list.config - A dictionary of keys and values of acceptable configuration parametersset_affinity method of the IENetwork class.get_supported_layers(net: IENetwork)
add_cpu_extenstion() method.net - A valid instance of IENetworkaffinity attribute of the IENetLayer class.This class represents a network instance loaded to plugin and ready for inference.
There is no explicit class constructor. To make a valid instance of ExecutableNetwork, use load() method of the IEPlugin class.
requests - A tuple of InferRequest instancesinfer(inputs=None)infer() method of the InferRequest classinputs - A dictionary that maps input layer names to numpy.ndarray objects of proper shape with input data for the layernumpy.ndarray objects with output data of the layerclassification_sample.py).start_async(request_id, inputs=None)async_infer() method of the InferRequest class.request_id - Index of infer request to start inferenceinputs - A dictionary that maps input layer names to numpy.ndarray objects of proper shape with input data for the layerInferRequest class.For more details about infer requests processing, see classification_sample_async.py (simplified case) and object_detection_demo_ssd_async.py (real asynchronous use case) samples.
get_exec_graph_info()IENetworkget_metric(metric_name: str)get_config(metric_config: str)This class provides an interface to infer requests of ExecutableNetwork and serves to handle infer requests execution and to set and get output data.
There is no explicit class constructor. To make a valid InferRequest instance, use load() method of the IEPlugin class with specified number of requests to get ExecutableNetwork instance which stores infer requests.
inputs - A dictionary that maps input layer names to numpy.ndarray objects of proper shape with input data for the layeroutputs - A dictionary that maps output layer names to numpy.ndarray objects with output data of the layer
Usage example:
It is not recommended to run inference directly on InferRequest instance. To run inference, please use simplified methods infer() and start_async() of ExecutableNetwork.
infer(inputs=None)inputs - A dictionary that maps input layer names to numpy.ndarray objects of proper shape with input data for the layerasync_infer(inputs=None)inputs - A dictionary that maps input layer names to numpy.ndarray objects of proper shape with input data for the layerwait(timeout=-1)NOTE: There are special values of the timeout parameter:
timeout - Time to wait in milliseconds or special (0, -1) cases described above. If not specified, timeout value is set to -1 by default.async_infer() method of the the InferRequest class.get_perf_counts()NOTE: Performance counters data and format depends on the plugin
set_batch(size)NOTE: Support of dynamic batch size depends on the target plugin.
batch - New batch size to be used by all the following inference calls for this requestset_completion_callback(py_callback, py_data = None)py_callback - Any defined or lambda functionpy_data - Data that is passed to the callback function