eprintid: 8456
rev_number: 2
eprint_status: archive
userid: 1
dir: disk0/00/00/84/56
datestamp: 2023-11-09 16:20:21
lastmod: 2023-11-09 16:20:21
status_changed: 2023-11-09 16:12:42
type: conference_item
metadata_visibility: show
creators_name: Essam, M.
creators_name: Tang, T.B.
creators_name: Ho, E.T.W.
creators_name: Chen, H.
title: Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training
ispublished: pub
keywords: Digital arithmetic, Deep belief network (DBN); Deep belief networks; Fixed points; Floating points; Mnist database; Precision arithmetic; Rounding algorithm; Scaling factors, Stochastic systems
note: cited By 7; Conference of 8th International IEEE EMBS Conference on Neural Engineering, NER 2017 ; Conference Date: 25 May 2017 Through 28 May 2017; Conference Code:129986
abstract: This paper reports how to train a Deep Belief Network (DBN) using only 8-bit fixed-point parameters. We propose a dynamic-point stochastic rounding algorithm that provides enhanced results compared to the existing stochastic rounding. We show that by using a variable scaling factor, the fixed-point parameter updates are enhanced. To be more hardware amenable, the use of common scaling factor at each layer of DBN is further proposed. Using publicly available MNIST database, we show that the proposed algorithm can train a 3-layer DBN with an average accuracy of 98.49, with a drop of 0.08 from the double floating-point average accuracy. © 2017 IEEE.
date: 2017
publisher: IEEE Computer Society
official_url: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85028585216&doi=10.1109%2fNER.2017.8008430&partnerID=40&md5=3d565633d6b0ee148b8349a9b15f1d76
id_number: 10.1109/NER.2017.8008430
full_text_status: none
publication: International IEEE/EMBS Conference on Neural Engineering, NER
pagerange: 629-632
refereed: TRUE
isbn: 9781538619162
issn: 19483546
citation:   Essam, M. and Tang, T.B. and Ho, E.T.W. and Chen, H.  (2017) Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training.  In: UNSPECIFIED.