video.ai

opencv-yolo/doc/README.md

+# 视频分析接口设计
+
+### 子系统组成与交互
+- mgr-svc: 管控中心后端
+- video-crud-svc: 视频文件CRUD微服务, 包括: 事件视频上传微服务, 视频查询微服务, 视频录制微服务
+- videoML-API-svc: 视频分析restful api
+- videoML-Worker: 视频分析异步任务处理器
+- azure-storage: 视频云存储
+
+![sd-video-ml](videml-sequce.png)
+
+### 接口定义
+- [videoML-API-svc API spec (OpenAPI 3.0)](video.ai.api.yml)
+- [videoML-API-svc MQTT spec](video.ai.mq.yml)
+
+
+### 产品业务流程
+1. **mgr-svc** 通过调用 videoML-API-svc的 va_config接口(/config_ipc)设定一组摄像头**开启**或**关闭**哪些视频分析功能
+在 video.ai.api.yml里可以看到 post body.
+2. **video-crud-svc** 在生成视频后向MQTT topic **/video.ai/v1.0/task** 发送视频分析任务, 并在数据库中将此视频对应记录的 **va_result**空object中加入属性**vaStartTime=ts**
+3. **videoML** 系统收到此任务进行处理, 并将结果通过MQTT topic **/video.ai/v1.0/result** 通知video-crud-svc
+4. **video-crud-svc** 收到 **/video.ai/v1.0/result**发布消息后将对应的视频记录的**va_result**字段增加property **result= [result]**, 同时记录结束事件 **vaEndTime=ts**
+5. 前端业务部分:
+   1. 在query视频返回后, 
+      1. 如果**va_result**部分是个**空json**, 表明这个视频是整套系统部署之前的. 可以不用做任何特殊处理. 也可以加一个小context 按钮: “进行分析” (调用task http接口, 在api spec里)
+      2. 如果va_result部分只有**vaStartTime=ts**, 表明这个视频还在分析过程中, 可以在对应视频的UI处加合适的挂件装饰.(正在分析中)
+      3. 如果va_result部分有**result=object**, **vaStartTime**, **vaEndTime**, 则表明视频分析完成
+         1. 通过**result.code, result.msg** 判断分析是否成功及原因.
+         2. 通过**result.data**来决定如何显示视频.
+            1. 如果result.data.huamanDetect.found == true, 给该视频的UI处增加一个小人装饰, 
+            2. 如果result.data.xyzDetect.found == true, 给该视频的UI处增加一个xyz对应的装饰,
+            3. 如果没有任何检测结果是true: 则在该视频UI处显示删除按钮装饰.
+
+
+问1: 为什么不直接删除视频, 如果没有检测到物体?
+
+答1: 因为目前不确定算法是否可靠. 并且物体检测只是AI的一个方面, 不能仅仅从这个方面来决定是否删除. AI检测只是提供更多关于视频的内容信息, 删不删除由产品业务决定. 不应该在基础服务里做这样的决定, 基础服务提供删除接口即可.
\ No newline at end of file

opencv-yolo/doc/video.ai.api.yml

+openapi: '3.0.0'
+info:
+  title: video.ai API
+  version: 0.0.1
+  description:
+    rootpath = /api/video.ai/v1.0
+tags:
+  - name: va_config
+  - name: task_api
+paths:
+  /config_ipc:
+    post:
+      summary: video analysis configuration for ip camera
+      tags:
+        - va_config
+      operationId: va_config
+      requestBody:
+        description: va_config
+        required: true
+        content:
+          application/json:
+            schema:
+              type: array
+              items:
+                $ref: '#/components/schemas/vaConfig'
+      responses:
+        '200':
+          description: ok
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/ok'
+        default:
+          description: error
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/error'
+  /task:
+    post:
+      summary: api for creating video.ai task synchronously. (there is another way by MQ.)
+      tags:
+        - task_api
+      operationId: none
+      requestBody:
+        description: task body
+        required: true
+        content:
+          application/json:
+            schema:
+              type: array
+              items:
+                $ref: '#/components/schemas/task'
+      responses:
+        '200':
+          description: ok
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/ok'
+        default:
+          description: error
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/error'
+components:
+  schemas:
+    vaConfig:
+      type: object
+      properties:
+        cameraId:
+          type: string
+          example: D72158932
+        features:
+          type: object
+          properties:
+            humanDetect:
+              type: object
+              properties:
+                enabled:
+                  type: boolean
+                thresh:
+                  description: confidence level. the bigger the stricter
+                  type: number
+                  maximum: 0.99
+                  minimum: 0.1
+                  default: 0.2
+    task:
+      type: object
+      properties:
+        cameraId:
+          type: string
+          example: D72158932
+          required: [endTime, startTime]
+        endTime:
+          type: integer
+          example: 1576842488000
+          required: [cameraId]
+        startTime:
+          type: integer
+          example: 1576842457000
+          required: [cameraId]
+        length:
+          type: integer
+          example: 200
+        image:
+          type: string
+          example: http://40.73.41.176/video/D72158932/1576842457000-1576842488999/firstFrame.jpg
+        video:
+          type: string
+          example: http://40.73.41.176/video/D72158932/1576842457000-1576842488999/1576842457000-1576842488999.mp4
+
+    ret:
+      type: object
+      required: [code, msg]
+      properties:
+        code:
+          type: integer
+        msg:
+          type: string
+        data:
+          type: object
+    ok:
+      type: object
+      $ref: '#/components/schemas/ret'
+      properties:
+        code:
+          type: integer
+          value: 0
+        msg:
+          type: string
+          value: 'ok'
+
+    error:
+      type: object
+      $ref: '#/components/schemas/ret'
+
+
+      
+
+
+        

opencv-yolo/doc/video.ai.mq.yml

+my_mq_spec: v1.0
+info:
+  title: MQ spec for video_crud_svc & video.ai
+  version: 0.0.1
+
+queue:
+- name: video.ai/v1.0/task
+  retian: false
+  type: shared-queue
+  sub-prefix: $queue/
+  pub-prefix: None
+  qos: 1
+  payload:
+    type: object
+    $ref: video.ai.api.yaml/components/schemas/task
+  producer: video_crud_svc
+  consumer: video.ai
+
+- name: video.ai/v1.0/result
+  retain: false
+  type: shared-queue
+  sub-prefix: $queue/
+  pub-prefix: None
+  qos: 1
+  payload:
+    type: object
+    properties:
+      code:
+        type: integer
+        description: 0 ok, otherwise failed
+      msg:
+        type: string
+        description: OK when code is 0. otherwise is application provided error message
+      target:
+          type: object
+          $ref: video.ai.api.yaml/components/schemas/task
+      data:
+          type: object
+          properties:
+            humanDetect:
+              type: object
+              properties:
+                found:
+                  type: boolean
+                image:
+                  type: string
+                  example:
+                    http://40.73.41.176/video/D72158932/1576842457000-1576842488999/firstFrame.jpg
+
+
+
+    
+
+

opencv-yolo/web/main/.editorconfig

+# editorconfig.org
+root = true
+
+[*]
+indent_style = space
+indent_size = 2
+end_of_line = lf
+charset = utf-8
+trim_trailing_whitespace = true
+insert_final_newline = true
+
+[*.md]
+trim_trailing_whitespace = false

opencv-yolo/web/main/.eslintrc.js

+module.exports = {
+  root: true,
+  env: {
+    browser: true,
+    node: true
+  },
+  parserOptions: {
+    parser: 'babel-eslint'
+  },
+  extends: [
+    '@nuxtjs',
+    'prettier',
+    'prettier/vue',
+    'plugin:prettier/recommended',
+    'plugin:nuxt/recommended'
+  ],
+  plugins: [
+    'prettier'
+  ],
+  // add your custom rules here
+  rules: {
+  }
+}

opencv-yolo/web/main/.gitignore

+# Created by .ignore support plugin (hsz.mobi)
+### Node template
+# Logs
+/logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+# Runtime data
+pids
+*.pid
+*.seed
+*.pid.lock
+
+# Directory for instrumented libs generated by jscoverage/JSCover
+lib-cov
+
+# Coverage directory used by tools like istanbul
+coverage
+
+# nyc test coverage
+.nyc_output
+
+# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
+.grunt
+
+# Bower dependency directory (https://bower.io/)
+bower_components
+
+# node-waf configuration
+.lock-wscript
+
+# Compiled binary addons (https://nodejs.org/api/addons.html)
+build/Release
+
+# Dependency directories
+node_modules/
+jspm_packages/
+
+# TypeScript v1 declaration files
+typings/
+
+# Optional npm cache directory
+.npm
+
+# Optional eslint cache
+.eslintcache
+
+# Optional REPL history
+.node_repl_history
+
+# Output of 'npm pack'
+*.tgz
+
+# Yarn Integrity file
+.yarn-integrity
+
+# dotenv environment variables file
+.env
+
+# parcel-bundler cache (https://parceljs.org/)
+.cache
+
+# next.js build output
+.next
+
+# nuxt.js build output
+.nuxt
+
+# Nuxt generate
+dist
+
+# vuepress build output
+.vuepress/dist
+
+# Serverless directories
+.serverless
+
+# IDE / Editor
+.idea
+
+# Service worker
+sw.*
+
+# Mac OSX
+.DS_Store
+
+# Vim swap files
+*.swp

opencv-yolo/web/main/.prettierrc

+{
+  "semi": false,
+  "arrowParens": "always",
+  "singleQuote": true
+}

opencv-yolo/web/main/README.md

+# main
+
+> My neat Nuxt.js project
+
+## Build Setup
+
+``` bash
+# install dependencies
+$ npm run install
+
+# serve with hot reload at localhost:3000
+$ npm run dev
+
+# build for production and launch server
+$ npm run build
+$ npm run start
+
+# generate static project
+$ npm run generate
+```
+
+For detailed explanation on how things work, check out [Nuxt.js docs](https://nuxtjs.org).

opencv-yolo/web/main/assets/README.md

+# ASSETS
+
+**This directory is not required, you can delete it if you don't want to use it.**
+
+This directory contains your un-compiled assets such as LESS, SASS, or JavaScript.
+
+More information about the usage of this directory in [the documentation](https://nuxtjs.org/guide/assets#webpacked).

opencv-yolo/web/main/components/Logo.vue

+<template>
+  <svg
+    class="NuxtLogo"
+    width="245"
+    height="180"
+    viewBox="0 0 452 342"
+    xmlns="http://www.w3.org/2000/svg"
+  >
+    <g fill="none" fill-rule="evenodd">
+      <path
+        d="M139 330l-1-2c-2-4-2-8-1-13H29L189 31l67 121 22-16-67-121c-1-2-9-14-22-14-6 0-15 2-22 15L5 303c-1 3-8 16-2 27 4 6 10 12 24 12h136c-14 0-21-6-24-12z"
+        fill="#00C58E"
+      />
+      <path
+        d="M447 304L317 70c-2-2-9-15-22-15-6 0-15 3-22 15l-17 28v54l39-67 129 230h-49a23 23 0 0 1-2 14l-1 1c-6 11-21 12-23 12h76c3 0 17-1 24-12 3-5 5-14-2-26z"
+        fill="#108775"
+      />
+      <path
+        d="M376 330v-1l1-2c1-4 2-8 1-12l-4-12-102-178-15-27h-1l-15 27-102 178-4 12a24 24 0 0 0 2 15c4 6 10 12 24 12h190c3 0 18-1 25-12zM256 152l93 163H163l93-163z"
+        fill="#2F495E"
+        fill-rule="nonzero"
+      />
+    </g>
+  </svg>
+</template>
+<style>
+.NuxtLogo {
+  animation: 1s appear;
+}
+
+@keyframes appear {
+  0% {
+    opacity: 0;
+  }
+  100% {
+    opacity: 1;
+  }
+}
+</style>

opencv-yolo/web/main/components/README.md

+# COMPONENTS
+
+**This directory is not required, you can delete it if you don't want to use it.**
+
+The components directory contains your Vue.js Components.
+
+_Nuxt.js doesn't supercharge these components._

opencv-yolo/web/main/layouts/README.md

+# LAYOUTS
+
+**This directory is not required, you can delete it if you don't want to use it.**
+
+This directory contains your Application Layouts.
+
+More information about the usage of this directory in [the documentation](https://nuxtjs.org/guide/views#layouts).

opencv-yolo/web/main/layouts/default.vue

+<template>
+  <div>
+    <nuxt />
+  </div>
+</template>
+
+<style>
+html {
+  font-family: 'Source Sans Pro', -apple-system, BlinkMacSystemFont, 'Segoe UI',
+    Roboto, 'Helvetica Neue', Arial, sans-serif;
+  font-size: 16px;
+  word-spacing: 1px;
+  -ms-text-size-adjust: 100%;
+  -webkit-text-size-adjust: 100%;
+  -moz-osx-font-smoothing: grayscale;
+  -webkit-font-smoothing: antialiased;
+  box-sizing: border-box;
+}
+
+*,
+*:before,
+*:after {
+  box-sizing: border-box;
+  margin: 0;
+}
+
+.button--green {
+  display: inline-block;
+  border-radius: 4px;
+  border: 1px solid #3b8070;
+  color: #3b8070;
+  text-decoration: none;
+  padding: 10px 30px;
+}
+
+.button--green:hover {
+  color: #fff;
+  background-color: #3b8070;
+}
+
+.button--grey {
+  display: inline-block;
+  border-radius: 4px;
+  border: 1px solid #35495e;
+  color: #35495e;
+  text-decoration: none;
+  padding: 10px 30px;
+  margin-left: 15px;
+}
+
+.button--grey:hover {
+  color: #fff;
+  background-color: #35495e;
+}
+</style>

opencv-yolo/web/main/middleware/README.md

+# MIDDLEWARE
+
+**This directory is not required, you can delete it if you don't want to use it.**
+
+This directory contains your application middleware.
+Middleware let you define custom functions that can be run before rendering either a page or a group of pages.
+
+More information about the usage of this directory in [the documentation](https://nuxtjs.org/guide/routing#middleware).

opencv-yolo/web/main/nuxt.config.js

+export default {
+  mode: 'universal',
+  /*
+   ** Headers of the page
+   */
+  head: {
+    title: process.env.npm_package_name || '',
+    meta: [
+      { charset: 'utf-8' },
+      { name: 'viewport', content: 'width=device-width, initial-scale=1' },
+      {
+        hid: 'description',
+        name: 'description',
+        content: process.env.npm_package_description || ''
+      }
+    ],
+    link: [{ rel: 'icon', type: 'image/x-icon', href: '/favicon.ico' }]
+  },
+  /*
+   ** Customize the progress-bar color
+   */
+  loading: { color: '#fff' },
+  /*
+   ** Global CSS
+   */
+  css: [],
+  /*
+   ** Plugins to load before mounting the App
+   */
+  plugins: [],
+  /*
+   ** Nuxt.js dev-modules
+   */
+  buildModules: [
+    // Doc: https://github.com/nuxt-community/eslint-module
+    '@nuxtjs/eslint-module'
+  ],
+  /*
+   ** Nuxt.js modules
+   */
+  modules: [
+    // Doc: https://bootstrap-vue.js.org
+    'bootstrap-vue/nuxt',
+    // Doc: https://axios.nuxtjs.org/usage
+    '@nuxtjs/axios',
+    '@nuxtjs/pwa',
+    // Doc: https://github.com/nuxt-community/dotenv-module
+    '@nuxtjs/dotenv'
+  ],
+  /*
+   ** Axios module configuration
+   ** See https://axios.nuxtjs.org/options
+   */
+  axios: {},
+  /*
+   ** Build configuration
+   */
+  build: {
+    /*
+     ** You can extend webpack config here
+     */
+    extend(config, ctx) {}
+  }
+}

opencv-yolo/web/main/package.json

+{
+  "name": "main",
+  "version": "1.0.0",
+  "description": "My neat Nuxt.js project",
+  "author": "burce.lu lzbgt@icloud.com",
+  "private": true,
+  "scripts": {
+    "dev": "nuxt",
+    "build": "nuxt build",
+    "start": "nuxt start",
+    "generate": "nuxt generate",
+    "lint": "eslint --ext .js,.vue --ignore-path .gitignore ."
+  },
+  "dependencies": {
+    "nuxt": "^2.0.0",
+    "bootstrap-vue": "^2.0.0",
+    "bootstrap": "^4.1.3",
+    "@nuxtjs/axios": "^5.3.6",
+    "@nuxtjs/pwa": "^3.0.0-0",
+    "@nuxtjs/dotenv": "^1.4.0"
+  },
+  "devDependencies": {
+    "@nuxtjs/eslint-config": "^1.0.1",
+    "@nuxtjs/eslint-module": "^1.0.0",
+    "babel-eslint": "^10.0.1",
+    "eslint": "^6.1.0",
+    "eslint-plugin-nuxt": ">=0.4.2",
+    "eslint-config-prettier": "^4.1.0",
+    "eslint-plugin-prettier": "^3.0.1",
+    "prettier": "^1.16.4"
+  }
+}

opencv-yolo/web/main/pages/README.md

+# PAGES
+
+This directory contains your Application Views and Routes.
+The framework reads all the `*.vue` files inside this directory and creates the router of your application.
+
+More information about the usage of this directory in [the documentation](https://nuxtjs.org/guide/routing).

opencv-yolo/web/main/pages/index.vue

+<template>
+  <div class="container">
+    <div>
+      <logo />
+      <h1 class="title">
+        main
+      </h1>
+      <h2 class="subtitle">
+        My neat Nuxt.js project
+      </h2>
+      <div class="links">
+        <a href="https://nuxtjs.org/" target="_blank" class="button--green">
+          Documentation
+        </a>
+        <a
+          href="https://github.com/nuxt/nuxt.js"
+          target="_blank"
+          class="button--grey"
+        >
+          GitHub
+        </a>
+      </div>
+    </div>
+  </div>
+</template>
+
+<script>
+import Logo from '~/components/Logo.vue'
+
+export default {
+  components: {
+    Logo
+  }
+}
+</script>
+
+<style>
+.container {
+  margin: 0 auto;
+  min-height: 100vh;
+  display: flex;
+  justify-content: center;
+  align-items: center;
+  text-align: center;
+}
+
+.title {
+  font-family: 'Quicksand', 'Source Sans Pro', -apple-system, BlinkMacSystemFont,
+    'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
+  display: block;
+  font-weight: 300;
+  font-size: 100px;
+  color: #35495e;
+  letter-spacing: 1px;
+}
+
+.subtitle {
+  font-weight: 300;
+  font-size: 42px;
+  color: #526488;
+  word-spacing: 5px;
+  padding-bottom: 15px;
+}
+
+.links {
+  padding-top: 15px;
+}
+</style>

opencv-yolo/web/main/plugins/README.md

+# PLUGINS
+
+**This directory is not required, you can delete it if you don't want to use it.**
+
+This directory contains Javascript plugins that you want to run before mounting the root Vue.js application.
+
+More information about the usage of this directory in [the documentation](https://nuxtjs.org/guide/plugins).

opencv-yolo/web/main/static/README.md

+# STATIC
+
+**This directory is not required, you can delete it if you don't want to use it.**
+
+This directory contains your static files.
+Each file inside this directory is mapped to `/`.
+Thus you'd want to delete this README.md before deploying to production.
+
+Example: `/static/robots.txt` is mapped as `/robots.txt`.
+
+More information about the usage of this directory in [the documentation](https://nuxtjs.org/guide/assets#static).

opencv-yolo/web/main/store/README.md

+# STORE
+
+**This directory is not required, you can delete it if you don't want to use it.**
+
+This directory contains your Vuex Store files.
+Vuex Store option is implemented in the Nuxt.js framework.
+
+Creating a file in this directory automatically activates the option in the framework.
+
+More information about the usage of this directory in [the documentation](https://nuxtjs.org/guide/vuex-store).

opencv-yolo/web/package-lock.json

+{
+  "requires": true,
+  "lockfileVersion": 1,
+  "dependencies": {
+    "pnpm": {
+      "version": "4.6.0",
+      "resolved": "https://registry.npmjs.org/pnpm/-/pnpm-4.6.0.tgz",
+      "integrity": "sha512-KTRmtPFUrRfSp3pC7nvg6+MD7KmLZFsnUVNmRgO/zOO/AUsTDIZsbwOqdxZULld7RUfZZmHbSHCGvudIX8Ix8w=="
+    }
+  }
+}

opencv-yolo/web/vamqtt.py

+import paho.mqtt.client as mqtt
+
+class VAMMQTTClient:
+  # The callback for when the client receives a CONNACK response from the server.
+  @staticmethod
+  def on_connect(client, userdata, flags, rc):
+      print("Connected with result code "+str(rc))
+
+      # Subscribing in on_connect() means that if we lose the connection and
+      # reconnect then subscriptions will be renewed.
+      client.subscribe("$queue/video.ai/v1.0/task")
+
+  # The callback for when a PUBLISH message is received from the server.
+  @staticmethod
+  def on_message(client, userdata, msg):
+      print(msg.topic+" "+str(msg.payload))
+      userdata(msg)
+
+  def __init__(self, callback, host='evcloud.ilabservice.cloud', port = 1883):
+    '''
+    Parameters
+    '''
+    self.client = mqtt.Client(userdata=callback)
+    self.client.on_connect = self.on_connect
+    self.on_message = self.on_message
+
+    self.client.connect(host, port, 30)
+    self.client.loop_start()
\ No newline at end of file

opencv-yolo/web/web.py

+#
+#
+#
+
+from flask import Flask, escape, request, jsonify, g, url_for
+from cerberus import schema_registry, Validator
+from celery import Celery
+import os, yaml, logging, time, datetime, threading
+from vamqtt import VAMMQTTClient
+
+"""
+va task schema:
+    {
+        "cameraId": "D72158932",
+        "endTime": 1576842488000,
+        "image": "http://40.73.41.176/video/D72158932/1576842457000-1576842488999/firstFrame.jpg",
+        "length": 260,
+        "startTime": 1576842457000,
+        "video": "http://40.73.41.176/video/D72158932/1576842457000-1576842488999/1576842457000-1576842488999.mp4"
+    }
+
+"""
+VA_SCHEMAS = {
+    'task': {
+        'cameraId': {'type': 'string', 'dependencies': ['startTime', 'endTime']},
+        'startTime': {'type': 'integer', 'dependencies':'cameraId'},
+        'endTime': {'type': 'integer', 'dependencies':'cameraId'},
+        'length': {'type': 'integer'},
+        'image': {'type': 'string'},
+        'video': {'type': 'string'}
+    },
+}
+
+app = Flask(__name__,
+            static_url_path='', 
+            static_folder='web/main/dist')
+logger = app.logger
+
+REDIS_ADDR = os.getenv('REDIS', 'redis://localhost:6379')
+app.config['broker_url'] = REDIS_ADDR
+app.config['result_backend'] = REDIS_ADDR
+worker = Celery(app.name, broker=app.config['broker_url'])
+worker.conf.update(app.config)
+worker.conf.update(
+    task_serializer='json',
+    #accept_content=['json'],
+    result_serializer='json',
+    #timezone='Europe/Oslo',
+    enable_utc=True)
+
+@app.route('/api/video.ai/v1.0/task', methods=['POST'])
+def new_task():
+    ret = {'code': 0,'msg': 'ok'}
+    taskValidator = Validator(VA_SCHEMAS['task'])
+    if not taskValidator.validate(request.json):
+        ret['code'] = 1
+        ret['msg'] = 'invalid request body'
+        ret['data'] = taskValidator.errors
+        return jsonify(ret)
+    else:
+        # process
+        video_analysis.apply_async(args=[request.json])
+        return jsonify(ret)
+
+@worker.task
+def video_analysis(data):
+    ret = {'code': 0, 'msg': 'ok'}
+    # get video
+    if 'cameraId' in data: # azure storage
+        pass
+    elif 'video' in data: # http
+        pass
+    else: # no video
+        ret['code'] = 1
+        ret['msg'] = 'no video specified'
+        return ret
+
+    # analyze
+    logger.info("aaaa")
+    return 'aaa'
+
+
+if __name__ == '__main__':
+    mq = VAMMQTTClient(new_task)
+    app.run(host='0.0.0.0', port = '5000')
+
+