Inference Model

inference_model(self, input_model_path: str | Path, target_device_name: Device | List[Device], inputs: Dataset | Mapping[str, List[ndarray]] | str, job_name: str | None = None, options: InferenceOptions | str = InferenceOptions(compute_unit=None, dequantize_outputs=True, tflite_delegates=None, tflite_options=None, qnn_options=None, onnx_options=None, onnx_execution_providers=None, max_profiler_iterations=100, max_profiler_time=600), retry: bool = True) → InferenceJob | List[InferenceJob]

Inference a model on a device in the QAI hub.

Parameters:

• input_model_path – The path to the input model.

• target_device_name – The device to benchmark the model on.

• inputs – The input data to use for the inference.

• job_name – The name of the job.

• options – The options to use for the inference.

• retry – Whether to retry the inference if it fails.

Returns:

Returns an inference job object if successful.

Return type:

Union[InferenceJob, List[InferenceJob]]

Note

For details, see submit_inference_job in QAI Hub API.