一个自定义插件需要实现两个类，分别继承于
class MyPluginDynamic : public IPluginV2DynamicEx
class MyPluginDynamicCreator : public IPluginCreator
然后使用
REGISTER_TENSORRT_PLUGIN(MyPluginDynamicCreator);
看其详细定义，也就是将这个插件加入到一个全局链表中，方便trtexec根据op name来查找该插件，这个实现跟ffmpeg中注册编解码器是一个思路
#define REGISTER_TENSORRT_PLUGIN(name) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ??
? ? static nvinfer1::PluginRegistrar<name> pluginRegistrar##name {}
#endif // NV_INFER_RUNTIME_H
下面是method详细的调用过程：
Creator:
MyPluginDynamicCreator
getPluginName （连续被调用两次）
getPluginNamespace
getPluginVersion
================================
Plugin:
getPluginVersion
getPluginNamespace
getPluginNamespace
getFieldNames
createPlugin
MyPluginDynamic
getNbOutputs
getOutputDataType
getNbOutputs
clone
MyPluginDynamic
setPluginNamespace
getPluginType
destroy
~MyPluginDynamic
getOutputDimensions
getOutputDataType
getOutputDimensions
?
=====================
一个有意义的参考
GitHub - eweill-nv/dcnv2_trt: TensorRT plugin forDCNv2 layer in ONNX model
IPluginV2DynamicExt
The first thing that we want to point out is the we are going to base our Plugin off of the IPluginV2DynamicExt class which will give us the ability to use alot of the functionality that TensorRT already has built in. You can see where we built our plugin class around the IPluginV2DynamicExt class here.
The first thing we want to do is to create our constructor and destructor for our TensorRT plugin (in this case, DCNv2PluginDynamic). You can see an example of that here:
DCNv2PluginDynamic();
DCNv2PluginDynamic(const void* data, size_t length, const std::string& name);
DCNv2PluginDynamic(DCNv2Parameters param, const std::string& name);
~DCNv2PluginDynamic() override;
Note that here, we have 2 different ways that a DCNv2PluginDynamic can be created: either passing in the data as a pointer and reading each value separately, or simply passing it in as a mParam structure with all of the data already included in the right format.
Methods
We have a few methods that are part of the IPluginV2DynamicExt class that we want to override so that we can modify if necessary:
clone(): copies over all internal plugin parameters and returns a new plugin object with these parameters
getOutputDimensions(): computes the dimensions of the output tensor from dimensions of the input tensor
supportsFormatCombination(): determines supported data types
configurePlugin(): configure the layer
getWorkspaceSize(): find the workspace size required by the layer (it is still necessary to provide the --workspace flag to trtexec as well)
enqueue(): execute the layer
Next, we have a few of the methods that are part of IPluginv2Ext that we want to override as well for our functionality:
getOutputDataType(): returns the datatype of the plugin output (in this case, either kFLOAT or kHALF)
attachToContext(): attach the plugin to an execution context and graph plugin access to context resources (use of cuBLAS/cuDNN/etc.)
detatchFromContext(): detach the plugin from its execution context
Lastly, we have a few of the methods that are part of IPluginV2 that we want to override for the same reason:
getPluginType(): return the type for the plugin (matches the plugin name returned by the plugin creator)
getPluginVersion(): returns the plugin version (should also match the plugin version returned by the plugin creator)
getNbOutputs(): returns number of outputs for the layer
initialize(): initialize the layer for execution (called when the engine is created)
terminate(): releases resources aqcuired during plugin layer initialization (called when engine is destroyed)
getSerializationSize(): returns size of serialization buffer necessary
serialize(): serialize the layer
destroy(): destroy the plugin object
setPluginNamespace(): set the namespace for the plugin object
getPluginNamespace(): return the namespace for the plugin object
More information about these plugins can be found here in the TensorRT documentation.
Members
As part of our IPluginV2DynamicExt instantiated class, we also want to create a few member variables that will help us with our plugin implementation. These can be found here
We have the following variables defined here:
mLayerName: given name for the layer (how it shows up in the graph)
mNamespace: namespace in which the layer resides
cublasHandle_: handle to the cuBlas context
cudnnHandle_: handle to the cuDNN context
mType: layer type (in this case, either kFLOAT or kHALF)
input1_shape: shape of first input to DCNv2 layer (from Add layer)
input2_shape: shape of second input to DCNv2 layer (from Conv layer)
weights_shape: shape of weights for DCNv2 layer
output_shape: shape of output for DCNv2 layer
mParam: structure container attributes for DCNv2 layer
mDeviceWeights: variable for weights on GPU for DCNv2 layer
mDeviceBiases: variable for biases on GPU for DCNv2 layer
IPluginCreator
For user implemented layers, we need to also instantiate another class (i.e. DCNv2PluginDynamicCreator) which is going to be part of the IPluginCreator class with the following methods:
The first thing we want to do (as before) is create our constructor and destructor for our TensorRT plugin creator (in this case, DCNv2PluginDynamicCreator). An example of that can be found here with the declaration here.
Methods
getTensorRTVersion(): return version of API the plugin creator was compiled with
getPluginName(): return plugin name
getPluginVersion(): return plugin version
getFieldNames(): return list of fields to be passed to createPlugin
createPlugin(): return plugin object
deserializePlugin(): called during deserialization of plugin layer
setPluginNamespace(): set namespace for plugin creator based on plugin library
getPluginNamespace(): return namespace of plugin creator object
Members
mFC: contains information about the PluginFieldCollection
mPluginAttributes: contains information about attributes of the plugin
mNamespace: namespace in which the layer reside
More information about the plugin creator can be found here in the TensorRT documentation.
