A Statistical Approach to Stochastic Computing Design and Analysis

Stochastic computing (SC) is an unconventional computing style that uses probabilistic bitstreams to implement algorithms like those for machine learning, digital filtering, and image processing. SC’s unusual encoding enables highly fault tolerant, low power and small datapaths. For instance, a single AND gate performs multiplication in SC. SC’s advantages make it attractive for use in small devices like wearables and biomedical implants, but widespread adoption of SC has been limited due to challenges including a steep accuracy-latency trade-off and expensive interface circuits. We address these challenges using a statistical approach. First, we develop a better framework for analyzing stochastic circuit errors and then use insights from statistical analysis to design and evaluate smaller and more accurate circuits.

Understanding SC’s many error sources is key to developing more accurate designs. However, existing approaches to accuracy analysis fail to provide a comprehensive account of all error types. The first part of this thesis proposes a novel framework named Bayesian Analysis of Stochastic Errors (BASE). BASE is built on statistical estimation theory and identifies three key quantities that must be modeled or estimated to quantify a circuit’s accuracy. BASE is used to better understand and address circuit errors and is compatible with prior methods for error analysis as well as new models that we propose in this thesis. Importantly, BASE reduces the reliance on black box simulation methods which provide limited insight and are prone to mistakes.

SC’s low-cost multipliers make it attractive for neural network and digital filtering applications. However, accurate addition is difficult to implement in SC and overhead from SC’s data conversion circuits reduces SC’s advantage for these applications. We address these problems by using insights from statistical modeling to develop new and better SC designs. Three such designs are introduced: 1) CeMux, a multiplexer adder that is significantly more accurate and smaller than its predecessors, 2) parallel sampling adders which have a flexible area-accuracy trade-off that can be tuned for the application, and 3) multiplexer-majority chains which can improve the overhead of SC’s conversion units. We apply our new designs to tasks like ECG filtering, image edge detection, and image classification with binarized neural networks and higher-precision neural networks.