You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
对于相同的FP16输入,进行了部分INT8量化的Convnext模型可以正确推理,但是对于非量化的FP16精度的Convnext模型,执行到类内的rknn_inputs_set()函数会报错“terminate called after throwing an instance of 'std::out_of_range'
what(): vector::_M_range_check: __n (which is 18446744073709551615) >= this->size() (which is 1)
已放弃”,由于librknn.so并不包含调试信息,我在GDB调试中无法具体定位到rknn_inputs_set()内部的问题,所以想请教下大家有没有遇到过类似的问题或者能提供一下解决的思路。
使用量化模型可以正确多线程推理并输出:
使用非量化模型相同输入下报错:
The text was updated successfully, but these errors were encountered:
对于相同的FP16输入,进行了部分INT8量化的Convnext模型可以正确推理,但是对于非量化的FP16精度的Convnext模型,执行到类内的rknn_inputs_set()函数会报错“terminate called after throwing an instance of 'std::out_of_range'
what(): vector::_M_range_check: __n (which is 18446744073709551615) >= this->size() (which is 1)
已放弃”,由于librknn.so并不包含调试信息,我在GDB调试中无法具体定位到rknn_inputs_set()内部的问题,所以想请教下大家有没有遇到过类似的问题或者能提供一下解决的思路。
使用量化模型可以正确多线程推理并输出:

使用非量化模型相同输入下报错:
The text was updated successfully, but these errors were encountered: