Task type

Topology name

Framework

Input blob sizes

Batch size

CPU: Intel(R) Core(TM) i3-8100 CPU @ 3.60GHz, CPU family: x86_64, GPU: Intel(R) Gen9 HD Graphics, RAM size: 32757916 kB, OS family: Linux, OS version: Linux-5.3.0-51-generic-x86_64-with-Ubuntu-18.04-bionic, Python version: 3.6.9

OpenVINO DLDT

CPU

FP32
Latency Mode	Throughput Mode

INT8
Latency Mode	Throughput Mode

GPU

FP32
Latency Mode

FP16
Latency Mode

CPU: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz, CPU family: x86_64, GPU: Intel(R) Gen9 HD Graphics, RAM size: 65727564 kB, OS family: Linux, OS version: Linux-5.3.0-51-generic-x86_64-with-Ubuntu-18.04-bionic, Python version: 3.6.9

OpenVINO DLDT

CPU

FP32
Latency Mode	Throughput Mode

INT8
Latency Mode	Throughput Mode

GPU

FP32
Latency Mode

FP16
Latency Mode

MYRIAD

FP16
Throughput Mode

Classification

alexnet

caffenet

densenet-121

densenet-161

densenet-169

densenet-201

googlenet-v1

googlenet-v2

googlenet-v3

googlenet-v4

inception-resnet-v2

mobilenet-v1-0.25-128

mobilenet-v1-0.50-160

mobilenet-v1-0.50-224

mobilenet-v1-1.0-224

mobilenet-v2

resnet-50

resnet-101

resnet-152

se-inception

se-resnet-50

se-resnet-101

se-resnet-152

se-resnext-50

se-resnext-101

squeezenet1.0

squeezenet1.1

vgg16

vgg19

Caffe

TensorFlow

Caffe

1x3x227x227

1x3x224x224

1x3x299x299

1x3x128x128

1x3x160x160

1x3x224x224

1x3x227x227

1x3x224x224

86.665

112.616

173.901

195.025

N/A

114.041

172.033

146.731

275.836

88.779

114.355

182.26

205.065

N/A

114.897

176.902

149.45

282.521

58.436

59.726

43.715

48.632

N/A

36.033

44.969

53.569

72.453

20.253

21.507

16.918

18.92

N/A

15.351

17.531

25.366

25.39

47.317

49.388

35.771

39.183

N/A

28.404

36.811

41.594

58.922

36.878

37.729

26.473

29.157

N/A

22.219

28.775

32.38

45.761

104.367

109.912

110.051

111.565

N/A

68.831

88.578

95.284

144.69

86.883

91.91

93.404

94.297

N/A

59.472

72.193

91.171

128.178

32.23

33.318

33.723

33.714

N/A

21.603

24.58

35.727

44.546

15.453

15.585

N/A

10.306

17.945

17.665

17.703

18.026

17.703

N/A

11.447

12.839

19.281

23.932

3043.762

4935.184

4941.743

5824.42

N/A

665.235

2240.846

698.701

2090.879

1110.78

1620.016

1504.683

1605.041

N/A

431.025

992.074

503.276

1403.335

687.422

899.803

743.127

846.15

N/A

335.853

561.186

395.465

805.919

252.35

288.395

271.395

274.698

N/A

162.683

200.208

242.151

378.928

278.895

314.241

320.736

327.823

N/A

156.475

190.757

206.149

360.277

50.846

52.772

55.16

54.001

N/A

37.069

43.096

64.219

83.976

25.71

26.293

27.543

26.86

N/A

18.606

21.323

32.828

42.062

17.282

17.54

18.359

17.898

N/A

12.454

14.145

22.071

28.009

76.723

82.03

77.285

81.231

N/A

54.794

67.667

81.125

119.278

36.615

36.924

33.625

36.535

N/A

29.982

33.159

50.039

63.078

19.667

20.121

18.551

19.883

N/A

15.701

17.418

26.102

33.354

13.529

13.662

12.584

13.412

N/A

10.633

11.701

17.675

22.462

30.231

30.377

27.781

29.952

N/A

24.038

24.886

40.906

38.596

17.514

17.637

16.451

17.176

N/A

14.184

14.836

23.593

21.629

200.56

205.546

190.837

202.132

N/A

128.885

145.756

186.356

252.268

400.947

437.916

378.509

423.267

N/A

219.563

282.89

283.322

460.009

11.223

12.4

13.054

13.046

N/A

10.062

10.517

24.661

19.795

9.192

9.946

10.408

10.33

N/A

8.033

8.333

20.617

15.777

105.271

141.572

269.161

296.86

N/A

130.639

195.132

178.686

310.976

60.268

61.854

107.296

142.946

276.98

293.387

N/A

132.121

199.374

180.152

317.012

60.266

60.472

87.086

98.093

69.324

66.15

N/A

40.748

50.941

59.769

81.17

39.457

40.366

31.459

34.127

26.252

27.027

N/A

17.79

19.849

28.923

28.733

11.042

11.114

69.259

80.429

57.068

53.268

N/A

32.149

41.936

46.92

65.935

29.579

29.742

53.564

63.078

42.157

38.603

N/A

25.283

32.721

37.066

51.347

20.878

20.904

159.098

184.505

177.746

183.413

N/A

79.43

101.128

106.682

163.007

93.209

95.468

131.698

148.615

150.598

150.266

N/A

66.881

81.665

101.39

144.511

87.376

90.066

50.433

53.68

53.625

54.232

N/A

24.354

27.898

39.571

50.387

20.481

21.393

24.264

25.297

N/A

11.678

20.105

10.449

27.519

28.423

28.904

28.233

N/A

12.946

14.588

21.572

27.148

9.066

9.131

3542.486

6853.505

7340.72

10719.17

N/A

764.547

2440.766

754.371

2321.142

364.513

646.206

1717.569

2729.767

2388.216

2717.209

N/A

537.766

1144.753

626.764

1540.961

282.454

340.071

1036.398

1592.179

1203.617

1253.424

N/A

375.296

646.695

497.014

927.841

190.305

205.846

367.647

476.862

427.6

456.514

N/A

177.432

227.799

261.882

427.783

117.388

123.289

406.78

515.771

486.857

489.342

N/A

170.051

217.495

225.403

421.48

74.335

77.005

76.488

84.67

87.219

87.554

N/A

42.181

48.848

72.219

95.054

37.788

37.903

38.738

41.91

44.166

43.118

N/A

21.108

24.185

36.861

47.557

16.794

19.786

26.159

27.914

29.513

28.618

N/A

14.065

16.056

24.661

31.712

10.892

13.269

115.114

135.607

129.815

126.973

N/A

61.697

76.782

91.061

134.205

46.123

47.086

56.959

59.941

51.292

52.89

N/A

34.414

37.827

55.935

71.751

19.91

19.726

30.601

33.241

29.447

29.677

N/A

17.821

19.896

29.174

38.357

10.408

10.312

20.986

22.776

20.188

20.144

N/A

12.06

13.338

19.836

25.833

7.194

7.075

47.014

50.076

42.603

43.567

N/A

27.013

28.019

45.401

43.28

15.291

15.382

26.979

30.02

26.132

25.566

N/A

15.83

16.698

26.156

24.555

8.702

8.728

332.828

354.113

305.54

315.49

N/A

143.628

166.104

208.558

282.685

162.232

169.209

693.273

779.82

601.452

634.637

N/A

240.451

321.181

334.434

509.802

226.612

302.959

17.0

19.251

20.774

20.57

N/A

11.397

11.965

28.39

22.633

9.844

10.274

14.064

15.459

16.586

16.284

N/A

9.102

9.482

23.681

17.951

8.128

8.079

Semantic segmentation

deeplabv3

TensorFlow

1x3x513x513

15.579

16.267

15.261

16.312

N/A

8.17

7.403

10.242

20.636

24.398

23.959

21.508

23.233

N/A

9.345

8.523

11.605

23.088

3.134

3.34

Object detection

ssd300

ssd512

mobilenet-ssd

ssd_mobilenet_v1_coco

ssd_mobilenet_v1_fpn_coco

ssd_mobilenet_v2_coco

Caffe

TensorFlow

1x3x300x300

1x3x512x512

1x3x300x300

1x3x640x640

1x3x300x300

6.353

6.147

6.334

5.897

N/A

4.996

5.186

10.736

8.907

2.186

2.062

2.177

2.041

N/A

1.922

1.828

3.627

3.363

134.201

141.885

136.469

141.712

N/A

80.951

98.955

116.463

168.889

115.466

125.112

115.402

125.641

N/A

66.575

81.007

92.236

132.285

3.087

3.159

3.067

3.164

N/A

2.401

2.436

4.206

4.228

74.797

84.471

80.911

86.205

N/A

41.375

50.003

62.714

95.943

10.313

10.226

10.261

10.192

N/A

5.731

5.924

12.317

10.216

4.001

4.0

3.546

3.516

3.511

3.503

N/A

2.198

2.082

4.182

3.841

1.349

1.362

195.493

229.709

200.525

219.757

N/A

90.882

112.425

135.589

190.456

56.812

57.398

173.587

200.506

169.109

183.14

N/A

74.734

91.583

100.585

149.477

45.199

41.601

4.977

4.989

4.858

4.956

N/A

2.738

2.783

4.777

4.812

113.994

133.792

123.528

134.403

N/A

46.769

57.346

69.564

108.13

30.715