• DeepSparkInference
• • Computer Vision
  • • Classification
    • Detection
    • Segmentation
    • Trace
  • NLP
  • • Language Model
  • Speech
  • • Speech Recognition
  • 社区
  • • 治理
    • 交流
    • 贡献
  • 许可证

DeepSparkInference

DeepSparkInference推理模型库作为DeepSpark开源社区的核心项目，于2024年3月正式开源，一期甄选了48个推理模型示例，涵盖计算机视觉，自然语言处理，语音识别等领域，后续将逐步拓展更多AI领域。

DeepSparkInference中的模型提供了在国产推理引擎IGIE或IxRT下运行的推理示例和指导文档，部分模型提供了基于国产通用GPU智铠100的评测结果。

IGIE（Iluvatar GPU Inference Engine）是基于TVM框架研发的高性能、高通用、全流程的AI推理引擎。支持多框架模型导入、量化、图优化、多算子库支持、多后端支持、算子自动调优等特性，为推理场景提供易部署、高吞吐量、低延迟的完整方案。

IxRT（Iluvatar CoreX RunTime）是天数智芯自研的高性能推理引擎，专注于最大限度发挥天数智芯通用GPU 的性能，实现各领域模型的高性能推理。IxRT支持动态形状推理、插件和INT8/FP16推理等特性。

DeepSparkInference将按季度进行版本更新，后续会逐步丰富模型类别并拓展大模型推理。

Computer Vision

Classification

Models	Precision	IGIE	IxRT
AlexNet	FP16	Supported	Supported
	INT8	Supported	Supported
CLIP	FP16	Supported	-
	INT8	-	-
Conformer-B	FP16	Supported	-
	INT8	-	-
CSPResNet50	FP16	-	Supported
	INT8	-	Supported
DenseNet121	FP16	Supported	-
	INT8	-	-
EfficientNet-B0	FP16	Supported	Supported
	INT8	-	Supported
EfficientNet_B1	FP16	-	Supported
	INT8	-	Supported
GoogLeNet	FP16	Supported	Supported
	INT8	Supported	Supported
HRNet-W18	FP16	Supported	-
	INT8	-	-
InceptionV3	FP16	Supported	-
	INT8	Supported	-
MobileNetV2	FP16	Supported	Supported
	INT8	Supported	Supported
MobileNetV3	FP16	-	Supported
	INT8	-	-
RepVGG	FP16	-	Supported
	INT8	-	-
Res2Net50	FP16	-	Supported
	INT8	-	-
ResNet101	FP16	-	Supported
	INT8	-	-
ResNet18	FP16	Supported	Supported
	INT8	Supported	Supported
ResNet34	FP16	-	Supported
	INT8	-	Supported
ResNet50	FP16	Supported	Supported
	INT8	Supported	-
ResNeXt50_32x4d	FP16	Supported	-
	INT8	Supported	-
ShuffleNetV1	FP16	-	Supported
	INT8	-	-
SqueezeNet 1.0	FP16	-	Supported
	INT8	-	Supported
Swin Transformer	FP16	Supported	-
	INT8	-	-
VGG16	FP16	Supported	Supported
	INT8	Supported	-

Detection

Models	Precision	IGIE	IxRT
RetinaNet	FP16	Supported	-
	INT8	-	-
YOLOv3	FP16	Supported	-
	INT8	Supported	-
YOLOv4	FP16	Supported	-
	INT8	Supported	-
YOLOv5	FP16	Supported	-
	INT8	Supported	-
YOLOv6	FP16	Supported	-
	INT8	-	-
YOLOv7	FP16	Supported	-
	INT8	Supported	-
YOLOv8	FP16	Supported	-
	INT8	Supported	-
YOLOX	FP16	Supported	Supported
	INT8	Supported	Supported

Segmentation

Models	Precision	IGIE	IxRT
Mask R-CNN	FP16	-	Supported
	INT8	-	-

Trace

Models	Precision	IGIE	IxRT
FastReID	FP16	Supported	-
	INT8	-	-
DeepSort	FP16	Supported	-
	INT8	Supported	-

NLP

Language Model

Models	Precision	IGIE	IxRT
BERT Base NER	FP16	-	-
	INT8	Supported	-
BERT Base SQuAD	FP16	Supported	Supported
	INT8	-	-
BERT Large SQuAD	FP16	Supported	Supported
	INT8	Supported	Supported

Speech

Speech Recognition

Models	Precision	IGIE	IxRT
Conformer	FP16	Supported	-
	INT8	-	-

---

社区

治理

请参见 DeepSpark Code of Conduct on Gitee or on GitHub。

交流

请联系 contact@deepspark.org.cn。

贡献

请参见 DeepSparkInference Contributing Guidelines。

许可证

本项目许可证遵循Apache-2.0。