Benchmark Model

benchmark_model(self, input_model_path: str | Path, target_device_name: Device | List[Device], options: ProfileOptions | str = ProfileOptions(compute_unit=None, dequantize_outputs=True, tflite_delegates=None, tflite_options=None, qnn_options=None, onnx_options=None, onnx_execution_providers=None, max_profiler_iterations=100, max_profiler_time=600), job_name: str | None = None, retry: bool = True) → BenchmarkerMetadata | List[BenchmarkerMetadata]

Benchmark a model on a device in the QAI hub.

Parameters:

• input_model_path – The path to the input model.

• target_device_name – The device to benchmark the model on.

• options – The options to use for the benchmark.

• job_name – The name of the job.

• retry – Whether to retry the benchmark if it fails.

Returns:

Returns a benchmarker metadata object if successful.

Return type:

Union[BenchmarkerMetadata, List[BenchmarkerMetadata]]

Note

For details, see submit_profile_job in QAI Hub API.

Example

Visit your job on Qualcomm AI Hub to see other inference metrics like memory usage, load time, layer by layer analysis and model visualization.

from netspresso import NPQAI
from netspresso.np_qai import Device
from netspresso.np_qai.options import ProfileOptions, TfliteOptions, ComputeUnit

QAI_HUB_API_TOKEN = "YOUR_QAI_HUB_API_TOKEN"
np_qai = NPQAI(api_token=QAI_HUB_API_TOKEN)

benchmarker = np_qai.benchmarker()

INPUT_MODEL_PATH = "YOUR_INPUT_MODEL_PATH"
OUTPUT_DIR = "YOUR_OUTPUT_DIR"
JOB_NAME = "YOUR_JOB_NAME"
DEVICE_NAME = "QCS6490 (Proxy)"

benchmark_options = ProfileOptions(
    compute_unit=[ComputeUnit.NPU],
    tflite_options=TfliteOptions(number_of_threads=4),
)

benchmark_result = benchmarker.benchmark_model(
    input_model_path=INPUT_MODEL_PATH,
    target_device_name=Device(DEVICE_NAME),
    options=benchmark_options,
    job_name=JOB_NAME,
)

# Monitor task status
while True:
    status = benchmarker.get_benchmark_task_status(benchmark_result.benchmark_task_info.benchmark_task_uuid)
    if status.finished:
        benchmark_result = benchmarker.update_benchmark_task(benchmark_result)
        print("Benchmark task completed")
        break
    else:
        print("Benchmark task is still running")