Manual Compression

Upload Model

upload_model(self, input_model_path: str, input_shapes: List[Dict[str, int]] | None = None, framework: Framework = Framework.PYTORCH) → ModelBase

Upload a model for compression.

Parameters:

input_model_path (str) – The file path where the model is located.
input_shapes (List[Dict[str, int]], optional) – Input shapes of the model. Defaults to [].
framework (Framework) – The framework of the model.

Raises:

e – If an error occurs while uploading the model.

Returns:

Uploaded model object.

Return type:

ModelBase

Details of Parameters

Framework

class Framework(value)[source]: An enumeration.

Available Framework

Name	Description
TENSORFLOW_KERAS	TensorFlow-Keras
PYTORCH	PyTorch GraphModule
ONNX	ONNX

Example

from netspresso.enums import Framework

FRAMEWORK = Framework.PYTORCH

Note

ONNX (.onnx)
- Supported version: PyTorch >= 1.11.x, ONNX >= 1.10.x.
- If a model is defined in PyTorch, it should be converted into the ONNX format before being uploaded.
- How-to-guide for the conversion of PyTorch into ONNX.
PyTorch GraphModule (.pt)
- Supported version: PyTorch >= 1.11.x.
- If a model is defined in PyTorch, it should be converted into the GraphModule before being uploaded.
- The model must contain not only the status dictionary but also the structure of the model (do not use state_dict).
- How-to-guide for the conversion of PyTorch into GraphModule.
TensorFlow-Keras (.h5, .zip)
- Supported version: TensorFlow 2.3.x ~ 2.8.x.
- Custom layer must not be included in Keras H5 (.h5) format.
- The model must contain not only weights but also the structure of the model (do not use save_weights).
- If there is a custom layer in the model, please upload TensorFlow SavedModel format (.zip).

Input Shapes

Note

For input shapes, use the same values that you used to train the model.
- If the input shapes of the model is dynamic, input shapes is required.
- If the input shapes of the model is static, input shapes is not required.

For example, batch=1, channel=3, height=768, width=1024.

input_shapes = [{"batch": 1, "channel": 3, "dimension": [768, 1024]}]

Currently, only single input models are supported.

Example

from netspresso import NetsPresso


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

compressor = netspresso.compressor_v2()
model = compressor.upload_model(
    input_model_path="./examples/sample_models/mobilenetv1.h5",
    input_shapes=[{"batch": 1, "channel": 3, "dimension": [224, 224]}],
)

Select Compression Method

select_compression_method(self, model_id: str, compression_method: ~netspresso.enums.compression.CompressionMethod, options: ~netspresso.clients.compressor.v2.schemas.compression.base.Options | None = Options(reshape_channel_axis=-1, policy=<Policy.AVERAGE: 'average'>, layer_norm=<LayerNorm.STANDARD_SCORE: 'standard_score'>, group_policy=<GroupPolicy.AVERAGE: 'average'>, step_size=2, step_op=<StepOp.ROUND: 'round'>, reverse=False)) → ResponseSelectMethod

Select a compression method for a model.

Parameters:

model_id (str) – The ID of the model.
compression_method (CompressionMethod) – The selected compression method.
options (Options, optional) – The options for pruning method.

Raises:

e – If an error occurs while selecting the compression method.

Returns:

The compression information for the selected compression method.

Return type:

ResponseSelectMethod

Details of Parameters

Compression Method

class CompressionMethod(value)[source]: An enumeration.

Available Compression Method

Name	Description
PR_L2	L2 Norm Pruning
PR_GM	GM Pruning
PR_NN	Nuclear Norm Pruning
PR_SNP	Structured Neuron-level Pruning
PR_ID	Pruning By Index
FD_TK	Tucker Decomposition
FD_SVD	Singular Value Decomposition
FD_CP	CP Decomposition

Example

from netspresso.enums import CompressionMethod

COMPRESSION_METHOD = CompressionMethod.PR_L2

Warning

Nuclear Norm is only supported in the Tensorflow-Keras framework.
Structured Neuron-level is only supported in the PyTorch and ONNX frameworks.

Note

Click on the link to learn more about the information. (Compression Method)

Options

class Policy(value)[source]: An enumeration.

class LayerNorm(value)[source]: An enumeration.

class GroupPolicy(value)[source]: An enumeration.

class StepOp(value)[source]: An enumeration.

Example

from netspresso.enums import Policy, LayerNorm, GroupPolicy
from netspresso.clients.compressor.v2.schemas import Options

OPTIONS = Options(
    policy=Policy.AVERAGE,
    layer_norm=LayerNorm.TSS_NORM,
    group_policy=GroupPolicy.COUNT,
    reshape_channel_axis=-1
)

Note

Click the link for more information. (Pruning Options)

Note

This parameter applies only to the Pruning Method (PR_L2, PR_GM, PR_NN, PR_SNP).

Details of Returns

Example

from netspresso import NetsPresso
from netspresso.enums import CompressionMethod, Policy, LayerNorm, GroupPolicy
from netspresso.clients.compressor.v2.schemas import Options


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

compressor = netspresso.compressor_v2()
compression_info = compressor.select_compression_method(
    model_id="YOUR_UPLOADED_MODEL_ID",
    compression_method=CompressionMethod.PR_L2,
    options=Options(
        policy=Policy.AVERAGE,
        layer_norm=LayerNorm.STANDARD_SCORE,
        group_policy=GroupPolicy.AVERAGE,
        reshape_channel_axis=-1,
    ),
)

Output

>>> compression_info
CompressionInfo(
    compression_method="PR_L2",
    available_layers=[
        AvailableLayer(name='conv1', values=[""], channels=[32]),
        AvailableLayer(name='layers.0.conv2', values=[""], channels=[64]),
        AvailableLayer(name='layers.1.conv2', values=[""], channels=[128]),
        AvailableLayer(name='layers.2.conv2', values=[""], channels=[128]),
        AvailableLayer(name='layers.3.conv2', values=[""], channels=[256]),
        AvailableLayer(name='layers.4.conv2', values=[""], channels=[256]),
        AvailableLayer(name='layers.5.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.6.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.7.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.8.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.9.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.10.conv2', values=[""], channels=[512]),
        AvailableLayer(name='layers.11.conv2', values=[""], channels=[1024]),
        AvailableLayer(name='layers.12.conv2', values=[""], channels=[1024])
    ],
    options={'reshape_channel_axis': -1, 'policy': 'average', 'layer_norm': 'tss_norm', 'group_policy': 'average'}
    original_model_id="YOUR_UPLOADED_MODEL_ID",
    compressed_model_id="",
    compression_id="",
)

Set Compression Params

Details of Parameters

Values of available layer

Compression Method	Number of Values	Type	Range
PR_L2	1	Float	0.0 < ratio < 1.0
PR_GM	1	Float	0.0 < ratio < 1.0
PR_NN	1	Float	0.0 < ratio < 1.0
PR_SNP	1	Float	0.0 < ratio < 1.0
PR_ID	(Num of Out Channels - 1)	Int	0 ≤ channels < Num of Out Channels
FD_TK	2	Int	0 < rank ≤ (Num of In Channels or Num of Out Channels)
FD_CP	1	Int	0 < rank ≤ min(Num of In Channels or Num of Out Channels)
FD_SVD	1	Int	0 < rank ≤ min(Num of In Channels or Num of Out Channels)

Example

for available_layer in compression_info.available_layers:
   available_layer.values = [0.2]

Output

>>> compression_info
CompressionInfo(
   compression_method="PR_L2",
   available_layers=[
      AvailableLayer(name='conv1', values=[0.2], channels=[32]),
      AvailableLayer(name='layers.0.conv2', values=[0.2], channels=[64]),
      AvailableLayer(name='layers.1.conv2', values=[0.2], channels=[128]),
      AvailableLayer(name='layers.2.conv2', values=[0.2], channels=[128]),
      AvailableLayer(name='layers.3.conv2', values=[0.2], channels=[256]),
      AvailableLayer(name='layers.4.conv2', values=[0.2], channels=[256]),
      AvailableLayer(name='layers.5.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.6.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.7.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.8.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.9.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.10.conv2', values=[0.2], channels=[512]),
      AvailableLayer(name='layers.11.conv2', values=[0.2], channels=[1024]),
      AvailableLayer(name='layers.12.conv2', values=[0.2], channels=[1024])
   ],
   options={'reshape_channel_axis': -1, 'policy': 'average', 'layer_norm': 'tss_norm', 'group_policy': 'average'}
   original_model_id="YOUR_UPLOADED_MODEL_ID",
   compressed_model_id="",
   compression_id="",
)

Compress Model

compress_model(self, compression: ResponseSelectMethod, output_dir: str, dataset_path: str | None = None) → CompressorMetadata

Compress a model using the provided compression information.

Parameters:

compression (CompressionInfo) – The information about the compression.
output_dir (str) – The local path to save the compressed model.
dataset_path (str, optional) – The path of the dataset used for nuclear norm compression method. Default is None.

Raises:

e – If an error occurs while compressing the model.

Returns:

Compress metadata.

Return type:

CompressorMetadata

Example

compressed_model = compressor.compress_model(
    compression=compression_info,
    output_dir="./outputs/compressed/graphmodule_manual",
)

Full Example

from netspresso import NetsPresso
from netspresso.enums import CompressionMethod, GroupPolicy, LayerNorm, Policy
from netspresso.clients.compressor.v2.schemas import Options


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

# 1. Declare compressor
compressor = netspresso.compressor_v2()

# 2. Upload model
model = compressor.upload_model(
    input_model_path="./examples/sample_models/graphmodule.pt",
    input_shapes=[{"batch": 1, "channel": 3, "dimension": [224, 224]}],
)

# 3. Select compression method
compression_info = compressor.select_compression_method(
    model_id=model.ai_model_id,
    compression_method=CompressionMethod.PR_L2,
    options=Options(
        policy=Policy.AVERAGE,
        layer_norm=LayerNorm.STANDARD_SCORE,
        group_policy=GroupPolicy.AVERAGE,
        reshape_channel_axis=-1,
    ),
)

# 4. Set params for compression(ratio or rank)
for available_layer in compression_info.available_layers[:5]:
    available_layer.values = [0.2]

# 5. Compress model
compressed_model = compressor.compress_model(
    compression=compression_info,
    output_dir="./outputs/compressed/graphmodule_manual",
)