Edge Trial Hardware
Updated: Oct 7
[NOTE: This post is now more than 2 years old, so of course the available hardware choices regarding the GPUs are different now. Also note that we no longer would recommend training on such small hardware, except for transfer training on very small data sets for narrow use cases.]
In this post we describe recommended minimal hardware setup for Edge Deployment. This can be used for a trial or for running small volume transcription or IVR. It will support at least a maximum of 25 concurrent Speech-to-Text (ASR) session. Note that: It is possible to run a trial on even less expensive hardware, see this blog post.
If the intention is to test Acoustic Model training, a single machine setup will work, but runtime recognition will not be possible as long as the training is in progress.
For a trial we recommend to use a workstation as it is easier find a workstation as opposed to rack server that can be configured with GPU at lower price points. Any standard rack type server may not work with a GPU, either for space or power (cabling) concerns, which necessitates purchase of a specific server that was built with GPU support capability. (We will soon post a blog with examples of rack servers that do support GPUs.)
An example workstation that will work out of the box is Dell Precision 5820 Tower. It is available directly from Dell with a variety of GPU configurations. Available GPUs range in price from a single NVIDIA® Quadro® RTX4000 for about $610 (we do not recommend cards with less than 6GB memory) all the way up to Dual Nvidia Quadro RTX8000 for over $13000.
For a basic trial the lowest spec CPU offered in Precision 5820 (i9-10900X), 64GB RAM and at least 1TB SSD storage will be sufficient. A Dell Precision 5820 configured like that will start at about $3500.
If the trial is to include testing of model training we will need more storage, at least 2TB SSD and a faster GPU, e.g., RTX5000. A Dell Precision 5820 configured like that will start at about $5000.
While the Nvidia GeForce consumer type GPUs have a price/performance advantage, one has to be aware of the limitations that Nvidia puts on consumer grade GPUs:
The warranty - for GeForce products Nvidia states: . Warranted Product is intended for consumer end user purposes only, and is not intended for datacenter use and/or GPU cluster commercial deployments ("Enterprise Use"). Any use of Warranted Product for Enterprise Use shall void this warranty.
The license - again, for GeForce products Nvidia states: No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.
Because, the recommendation here is for a Trial type of use and because we do recommend a workstation rather than a server, below we give a recommendation for both Consumer and Enterprise grade GPUs.
Consumer Grade Cards
For testing recognition (w/o training) single GeForce RTX 2070 (at $500) is more than sufficient.
For testing acoustic model training a set of two GeForce RTX 2080 Ti (at $1200 each) cards offer a good price to performance ratio. A faster choice would be dual Titan RTX cards but these are twice as expensive (about $5000 for two) with a speed advantage of about 30%.
Enterprise Grade Cards
For testing just the recognition Quadro RTX4000 (at $900) will provide sufficient performance although it will be a slower than RTX 2080 at similar price point.
For testing model training, we recommend at least a single Nvidia Quadro RTX6000 (at about $4000) but preferably two of them.