Please use this identifier to cite or link to this item:
Title: Machine Learning Using Approximate Computing
Authors: Sabir, Muhammad Dilshad
Keywords: Engineering & Technology
Computer Engineering
Issue Date: 2022
Publisher: National University of Science & Technology, Islamabad.
Abstract: Two-dimensional convolution/correlation is the ubiquitous tool to extract features for classification in a variety of machine learning and image processing applications such as CPR (Correlation Pattern Recognition), CNN (Convolution Neural Network), and filter processing. However, using these tools in Internet-of-Things (IoT)–based applications face stringent constraints, like limited memory capacity, inadequate computational re sources, and energy resources. The prime objective of this thesis is to propose a set of algorithms and techniques to reduce the computation workload due to an excessive num ber of correlation or convolution operations in CPRs and CNNs respectively. To achieve this objective, both CPR filters and CNN’s models require their approximated versions without any accuracy degradation. However, the research focuses on obtaining these ap proximated versions for future IoT implementation. This discretion makes the following contributions: For CNN, (a) to overcome the high computation cost of existing convolu tion algorithms, a hybrid algorithm is proposed that integrates the unique computational advantages of Winograd and spatial convolution, (b) a Particle of Swarm Convolution Layer Optimization (PSCLO) scaling is proposed that minimize accuracy loss and max imize the reduction in computational workload to combine both approximations, (c) an analysis of experimental results of symmetry and tile quantization approximation in conjunction with PSCLO is performed that finds the trade-off between the intensity of approximation and accuracy degradation. For CPR, (d) a Weight Quantization Retrain ing (WQR) approach is proposed to retrain low-precision quantization weights of the CPR filter for dynamic fixed point (DFP) and power-of-two(Po2) quantization schemes, additionally, the Particle of Swarm Optimization technique is employed to fine-tune performance parameters, (e) pre-processing strategies of log-polar and inverse log-polar transforms are used to support the low-precision CPR filter quantization, (f) analysis xi is performed to compare the advantages of spatially-filters (ST) and frequency-trained (FT) filters, this analysis is further extended to each domain, either spatially trained or frequency-trained, to investigate the comparative benefits of Po2 and DFP quantization schemes, (g) the overall analysis compares the advantages of direct, log-polar, inverse log-polar, and WQR, which provides a better perspective. For CNN’s, the proposed techniques and algorithms achieved ∼5.28x fewer multiplication operations without sig nificant accuracy loss on ResNet-18. For LeNet, that reduction is ∼3.87x and ∼3.93x on MNIST and Fashion-MNIST respectively. While the additive workload reductions for the above datasets were ∼2.5x and ∼2.56x respectively. For CIFAR-10 quick net work, the techniques acquire ∼9.28x and ∼8.82x fewer multiplication on CIFAR-10 and SVHN datasets. The additive workload reductions for these datasets are ∼1.70x and ∼1.33x respectively.For CPR filters, the following results are obtained for a common dataset. For the direct quantization approach, a compression ratio of 8 achieved 4.37x speedup without accuracy loss. However, a compression ratio of 4 with a log-polar im plementation achieved 1.12x speedup with 16% accuracy loss. Inverse log-polar with a compression ratio of 16 acquired 8.90x speedup with 6% accuracy loss. These empirical investigations demonstrate the effectiveness of the proposed approximation methods for both CPR and CNN using standard datasets.
Gov't Doc #: 25907
Appears in Collections:PhD Thesis of All Public / Private Sector Universities / DAIs.

Files in This Item:
File Description SizeFormat 
Muhammad Dilshad Sabir Computer Engineering 2022 nust isb.pdf 7.6.22.pdfphd.Thesis26.39 MBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.