What was done

I worked on building a Deep Learning Human Detection Exercise to identify the presence of humans and identification of the rectangular boundary around them. Apart from the live and video inference features, the exercise also includes model benchmarking and model visualization. The user is expected to upload a Deep Learning model which fits the required input and output specifications for inference. The input model is supposed to be in the ONNX format.
The project work also included compiling and testing an exercise guide for the user, which contains everything from fine tuning pre built object detection models in different frameworks(PyTorch and TensorFlow), to its subsequent conversion to the ONNX format.

Below is a very brief timeline of the work done. For more information and details, please refer to the specific week in the blog post.

Week Work Done
Community Bonding Period Hands on ONNX framework, Hands on AILIA-SDK, Study on object detection, Brushing up with the required tech stack, Fixing minor issues in existing DL exercises
Week 1 Building basic DL Human Detection exercise(Choosing pre-built models, Exporting the model to ONNX format, Preprocessing the input, Running inference, Postprocessing the outputs)
Week 2 Added model analyzer, Started working on model benchmarking, Collecting benchmarking dataset, Preprocessing the dataset
Week 3 Continuing with the benchmarking code, Benchmarking on the Oxford Town Center dataset, Added custom buttons and their functionalities for benchmarking and visualization
Week 4 Documented guides to train and fine tune pre-existing DL models in PyTorch and TensorFlow. This includes everything from making the process of collecting data, preprocessing it, fine tuning with the data on a pre-existing model architecture and converting the model to the ONNX format.
Week 5 Added video inference feature to the exercise, Made template changes to make exercise look minimalistic, Added benchmarking results and graph on the exercise console and canvas
Week 6 Implemented the upload of model/video an independent event from executing an exercise mode, Worked on the base Deep Learning server template for the exercise, Worked on the main exercise template used during the docker execution, Fixed exercise connection issue, Added the new noVNC console to the web template, Implemented more specific activation/deactivation of buttons based on wether the required files have been fully uploaded or not
Week 7 Resolved issue of process thread closing when launching different modes one after another, Resolved benchmarking issue occurring during the docker launch via start.sh, Resolved model visualizer connectivity issue occurring during the docker launch, Resolved output re-direction issue to the NoVnc console which arose after the one RADI launch per session update
Week 8 University Exam break
Week 9 Worked on GPU integration for the exercise(Dependencies included onnxruntime-gpu, CUDA runtime, CUDNN, and supported Nvidia drivers)
Week 10 Continued the work on GPU integration, Cleaning up the code base

Working Demo

To Do

  • Providing support for GPU and CPU inference from a common docker image

Merged PR’s