Vitis AIの構造を勉強してみる(9日目)

先日、諦めたライブラリをどうにかしてみよう。

ライブラリのソースコードを覗く

改めて、ソースコードの構成を確認する。

$ cd Vitis-AI/tools/Vitis-AI-Library/facedetect
$ tree
.
├── CMakeLists.txt
├── include
│   └── vitis
│       └── ai
│           └── facedetect.hpp
├── src
│   ├── detect_imp.cpp
│   ├── detect_imp.hpp
│   └── facedetect.cpp
└── test
    ├── test_accuracy_facedetect.cpp
    ├── test_facedetect.cpp
    └── test_facedetect_batch.cpp

testはライブラリのテスト用だと思うのでsrcだけビルドできればいいだろう。

ファイルは3つなので全部、覗いてみよう。

まずはfacedetect.cppを見る。

/*
 * Copyright 2019 Xilinx Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
#include <vitis/ai/facedetect.hpp>

#include "./detect_imp.hpp"

namespace vitis {
namespace ai {

FaceDetect::FaceDetect() {}
FaceDetect::~FaceDetect() {}
std::unique_ptr<FaceDetect> FaceDetect::create(const std::string &model_name,
                                               bool need_preprocess) {
  return std::unique_ptr<FaceDetect>(
      new DetectImp(model_name, need_preprocess));
}
std::unique_ptr<FaceDetect> FaceDetect::create(const std::string &model_name,
                                               xir::Attrs *attrs,
                                               bool need_preprocess) {
  return std::unique_ptr<FaceDetect>(
      new DetectImp(model_name, attrs, need_preprocess));
}

}  // namespace ai
}  // namespace vitis

これが本体だよね。

先日はvitis::ai::main_for_jpeg_demoがハードウェアだよねと位置付けたが実行するアプリのtest_jpeg_facedetect.cppvitis::ai::FaceDetect::create(model)でこのソースコードを呼び出すということですね。

ここから、createされてDetecgImp関数を呼び出すわけですね。

あれ?

main_for_jpeg_demoのソースコードがない。

ここではなくて別のところにあるんだな。

main_for_jpeg_demoVitis-AI/tools/Vitis-AI-Library/benchmark/include/vitis/ai/demo.hppに次のようにいた。

// Entrance of jpeg demo
template <typename FactoryMethod, typename ProcessResult>
int main_for_jpeg_demo(int argc, char *argv[],
                       const FactoryMethod &factory_method,
                       const ProcessResult &process_result, int start_pos = 1) {
  if (argc <= 1) {
    usage_jpeg(argv[0]);
    exit(1);
  }
  auto model = factory_method();
  for (int i = start_pos; i < argc; ++i) {
    auto image_file_name = std::string{argv[i]};
    auto image = cv::imread(image_file_name);
    if (image.empty()) {
      LOG(FATAL) << "cannot load " << image_file_name << std::endl;
      abort();
    }
    auto result = model->run(image);
    image = process_result(image, result, true);
    auto out_file =
        image_file_name.substr(0, image_file_name.size() - 4) + "_result.jpg";
    cv::imwrite(out_file, image);
    LOG_IF(INFO, ENV_PARAM(DEBUG_DEMO)) << "result image write to " << out_file;
  }
  LOG_IF(INFO, ENV_PARAM(DEBUG_DEMO)) << "BYEBYE";
  return 0;
}

}  // namespace ai
}  // namespace vitis

ここでOpenCVを利用してJPEGをimreadで読み込み、model->run(image)で実行して、process_resultでなにかして、imwriteで結果を書き戻しているんだな。

process_resultはfacedetectのディレクトリにdemo/Vitis-AI-Library/samples/facedetect/process_result.hppがいたな。

こんなかんじだな。

/*
 * Copyright 2019 Xilinx Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
#include <iostream>
#include <opencv2/opencv.hpp>
#include <string>

cv::Mat process_result(cv::Mat &m1, const vitis::ai::FaceDetectResult &result,
                       bool is_jpeg) {
  cv::Mat image;
  cv::resize(m1, image, cv::Size{result.width, result.height});
  for (const auto &r : result.rects) {
    LOG_IF(INFO, is_jpeg) << " " << r.score << " "  //
                          << r.x << " "             //
                          << r.y << " "             //
                          << r.width << " "         //
                          << r.height;
    cv::rectangle(image,
                  cv::Rect{cv::Point(r.x * image.cols, r.y * image.rows),
                           cv::Size{(int)(r.width * image.cols),
                                    (int)(r.height * image.rows)}},
                  0xff);
  }

  return image;
}

結果を四角で書き込むのか。

次は実行本体のDetectImpを見てみよう。

ソースコードはdetect_imp.cppを見る。

/*
 * Copyright 2019 Xilinx Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
#include "detect_imp.hpp"

#include <vitis/ai/profiling.hpp>

using std::vector;

namespace vitis {
namespace ai {

DetectImp::DetectImp(const std::string &model_name, bool need_preprocess)
    : vitis::ai::TConfigurableDpuTask<FaceDetect>(model_name, need_preprocess),
      det_threshold_(configurable_dpu_task_->getConfig()
                         .dense_box_param()
                         .det_threshold()) {  //
}

DetectImp::DetectImp(const std::string &model_name, 
                     xir::Attrs *attrs,
                     bool need_preprocess)
    : vitis::ai::TConfigurableDpuTask<FaceDetect>(model_name, attrs,
                                                  need_preprocess),
      det_threshold_(configurable_dpu_task_->getConfig()
                         .dense_box_param()
                         .det_threshold()) {  //
}

DetectImp::~DetectImp() {}

FaceDetectResult DetectImp::run(const cv::Mat &input_image) {
  __TIC__(FACE_DETECT_E2E)
  // Set input image into DPU Task
  cv::Mat image;
  auto size = cv::Size(getInputWidth(), getInputHeight());
  if (size != input_image.size()) {
    cv::resize(input_image, image, size, 0);
  } else {
    image = input_image;
  }
  __TIC__(FACE_DETECT_SET_IMG)
  configurable_dpu_task_->setInputImageBGR(image);
  __TOC__(FACE_DETECT_SET_IMG)

  __TIC__(FACE_DETECT_DPU)
  configurable_dpu_task_->run(0);
  __TOC__(FACE_DETECT_DPU)

  __TIC__(FACE_DETECT_POST_ARM)
  auto ret = vitis::ai::face_detect_post_process(
      configurable_dpu_task_->getInputTensor(),
      configurable_dpu_task_->getOutputTensor(),
      configurable_dpu_task_->getConfig(), det_threshold_);
  __TOC__(FACE_DETECT_POST_ARM)

  __TOC__(FACE_DETECT_E2E)
  return ret[0];
}

std::vector<FaceDetectResult> DetectImp::run(
    const std::vector<cv::Mat> &input_images) {
  __TIC__(FACE_DETECT_E2E)
  // Set input image into DPU Task
  std::vector<cv::Mat> images;
  auto size = cv::Size(getInputWidth(), getInputHeight());
  for (auto i = 0u; i < input_images.size(); i++) {
    if (size != input_images[i].size()) {
      cv::Mat img;
      cv::resize(input_images[i], img, size, 0);
      images.push_back(img);
    } else {
      images.push_back(input_images[i]);
    }
  }
  __TIC__(FACE_DETECT_SET_IMG)
  configurable_dpu_task_->setInputImageBGR(images);
  __TOC__(FACE_DETECT_SET_IMG)

  __TIC__(FACE_DETECT_DPU)
  configurable_dpu_task_->run(0);
  __TOC__(FACE_DETECT_DPU)

  __TIC__(FACE_DETECT_POST_ARM)
  auto ret = vitis::ai::face_detect_post_process(
      configurable_dpu_task_->getInputTensor(),
      configurable_dpu_task_->getOutputTensor(),
      configurable_dpu_task_->getConfig(), det_threshold_);
  __TOC__(FACE_DETECT_POST_ARM)

  __TOC__(FACE_DETECT_E2E)
  return ret;
}

float DetectImp::getThreshold() const {
  std::lock_guard<std::mutex> lock(mtx_threshold_);
  return det_threshold_;
}

void DetectImp::setThreshold(float threshold) {
  std::lock_guard<std::mutex> lock(mtx_threshold_);
  det_threshold_ = threshold;
}

}  // namespace ai
}  // namespace vitis

ソースコードの長さからするとこいつが本命のようだな。

最後にヘッダファイルのdetect_imp.hppも確認しておくか。

/*
 * Copyright 2019 Xilinx Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
#pragma once
#include <mutex>
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <vector>
#include <vitis/ai/configurable_dpu_task.hpp>
#include <vitis/ai/facedetect.hpp>
using std::tuple;
using std::vector;

namespace vitis {
namespace ai {

class DetectImp : public vitis::ai::TConfigurableDpuTask<FaceDetect> {
 public:
  DetectImp(const std::string &model_name, bool need_preprocess);
  DetectImp(const std::string &model_name, xir::Attrs *attrs, bool need_preprocess);

  /// Destructor
  virtual ~DetectImp();
  /// Set an image and get positions and scores of faces in the image
  virtual FaceDetectResult run(const cv::Mat &img) override;

  /// Set an image list and get positions and scores of faces in the image
  virtual std::vector<FaceDetectResult> run(
      const std::vector<cv::Mat> &img) override;
  /// Get detect threshold
  virtual float getThreshold() const override;
  /// Set detect threshold
  virtual void setThreshold(float threshold) override;

 private:
  float det_threshold_;
  mutable std::mutex mtx_threshold_;
};

}  // namespace ai
}  // namespace vitis

detectImpTConfigurableDpuTaskdetect_imp.hppにいたのか。

そして、det_threshold_floatなのね。

そういえば、ライブラリを確認したいんじゃなくって、ライブラリをビルドする方法を確認するんだった。

明日にしよう。

write: 2021/01/16/ 00:00:00