Enterprises are increasingly looking to mine the treasure trove of insights from voice conversations using AI. These conversations take place daily on video meeting platforms like Zoom, Google Meet and Microsoft Teams and over telephony in the contact center (which take place on CCaaS or on-premise contact center telephony platforms).

What is Voice AI?

Voice AI or Conversational AI refers to converting the audio from these conversations into text using Speech recognition/ASR technology and mining the transcribed text for analytics and insights using NLU. In addition to this, AI can be used to detect sentiment, energy and emotion in both the audio and text. The insights from NLU include extraction of key items from meetings. This include semantically matching phrases associated with things like action items. issues, sales blockers, agenda etc.

Over the last few years, the conversational AI space has seen many players launch highly successful products and scale their businesses. However most of these popular Voice AI options available in the market are multi-tenant SaaS offerings. They are deployed in a large public cloud provider like Amazon, Google or Microsoft. At first glance, this makes sense. Most enterprise software apps that automate  workflows in functional areas like Sales and Marketing(CRM), HR, Finance/Accounting or Customer service are architected as multi-tenant SaaS offerings. The move to Cloud has been a secular trend for business applications and hence Voice AI has followed this path.

However at Voicegain, we firmly believe that a different approach is required for a large segment of the market. We propose an Edge architecture using a single-tenant model is the way to go for Voice AI Apps.

Why does the Edge make sense for Conversational AI?

By Edge, we mean the following

1) The AI models for Speech Recognition/Speech-to-Text and NLU run on the customer's single tenant infrastructure – whether it is bare-metal in a datacenter or on a dedicated VPC with a cloud provider.

2) The Conversational AI app -which is usually a browser based application that uses these AI models is also completely deployed behind the firewall.

We believe that the advantages for Edge/On-Prem architecture for Conversational/Voice AI is being driven by the following four factors

1.    Privacy, Confidentiality and Data Residency requirements

Very often, conversations in meetings and call centers are sensitive from a business perspective. Enterprise customers in many verticals (Financial Services, Health Care, Defense, etc) are not comfortable storing the recordings and transcripts of these conversations on the SaaS Vendor's cloud infrastructure. Think about a highly proprietary information like product strategy, status of key deals, bugs and vulnerabilities in software or even a sensitive financial discussion prior to the releasing of earnings for a public company. Many countries also impose strict data residency requirements from a legal/compliance standpoint. This makes the Edge (On-Premises/VPC) architecture very compelling.

2. Accuracy/Model Customization

Unlike pure workflow-based SaaS applications, Voice AI apps include deep-learning based AI Models –Speech-to-Text and NLU. To extract the right analytics, it is critical that these AI models – especially the acoustic models in the speech-recognition/speech-to-text engine are trained on client specific audio data. This is because each customer use case has unique audio characteristics which limit the accuracy of an out-of-the-box multi-tenant model. These unique audio characteristics relate to

1.    Industry jargon – acronyms, technical terms

2.    Unique accents

3.    Names of brands, products, and people

4.    Acoustic environment and any other type of audio.

However, most AI SaaS vendors today use a single model to serve all their customers. And this results in sub-optimal speech recognition/transcription which in turn results in sub-optimal NLU. 

3. Latency ( for Real-time Voice AI apps) 

For real-time Voice AI apps - for e.g in the Call Center - there is an architectural advantage for the AI models to be in the same LAN as the audio sources.

4. Affordability

For many enterprises, SaaS Conversational AI apps are inexpensive to get started but they get very expensive at scale.

‍

Voicegain’s Edge Offering

Voicegain offers an Edge deployment where both the core platform and a web app like Voicegain Transcribe can operate completely on our clients infrastructure. Both can be placed "behind an enterprise firewall".

Most importantly Voicegain offers a training toolkit and pipeline for customers to build and train custom acoustic models that power these Voice AI apps.

Have a question? Or just want to talk?

If you have any question or you would like to discuss this in more detail, please contact our support team over email (support@voicegain.ai) 

‍

‍

As we announced here, Voicegain Transcribe is an AI based Meeting Assistant that you can take with you to all your work meetings. So irrespective of the meeting platform - Zoom, Microsoft Teams, Webex or Google Meet - Voicegain Transcribe has a way to support you.

‍

We now have some exciting news for those users that regularly host Zoom meetings. Voicegain Transcribe users who are on Windows now have a free, easy and convenient way to access all their meeting transcripts and notes from their Zoom meetings. Transcribe Users can now download a new client app that we have developed - Voicegain Zoom Meeting Assistant for Local Recordings - onto their device.

‍

With this client app, any Local Recording of a Zoom meeting (explained below) will be automatically submitted to Voicegain Transcribe. Voicegain's highly accurate AI models subsequently process the recording to generate both the transcript (Speech-to-Text) but also the minutes of the meeting and the topics discussed (NLU).

‍

As always, you get started with a free plan that does not expire. So you can get started today without having to setup your payment information.

‍

What is Zoom Local Recording?

Zoom provides two options to record meetings on its platform - 1) Local Recording and 2) Cloud Recording.

Zoom Local recording is a recording of the meeting that is saved on the hard disk of the user's device. There are two distinct benefits of using Zoom Local Recording

1. Free: Zoom offers this Local Recording feature even on free Zoom accounts. So you can try this feature even if you are on an unpaid Zoom Account
2. Privacy & Control: The audio content of your meeting could contain sensitive and confidential information. With a local recording, the audio is not shared with Zoom

Zoom Cloud Recording is when the recording of the meeting is stored on your Zoom Cloud account on Zoom's servers. Currently Voicegain does not directly integrate with Zoom Cloud Recording (however it is on our roadmap). In the interim, a user may download the Cloud Recording and upload it to Voicegain Transcribe in order to transcribe and analyze recordings saved in the cloud.

How does it work?

1. Sign up for a free account with Voicegain Transcribe. Here is a link to our sign up page. Pick the first option.

2. On the left menu click on Apps. You would visit a page as shown below

‍

3. Please refer to this knowledge-base article for steps after you download the Meeting Assistant.

Recording of individual speaker audio tracks

Zoom allows you to record individual speaker audio tracks separately as independent audio files. The screenshot above shows how to enable this feature on Zoom.

Voicegain Zoom Meeting Assistant for Local Recording supports uploading these independent audio files to Voicegain Transcribe so that you can get accurate speaker transcripts

Support for On-Premise/VPC and white-label UI

The entire Voicegain platform including the Voicegain Transcribe App and the AI models can be deployed On-Premise (or in VPC) giving an enterprise a fully secure meeting transcription and analytics offering.

Have a question?

If you have any question, please sign up today, and contact our support team using the App.

‍

Since June 2020, Voicegain has published benchmarks on the accuracy of its Speech-to-Text relative to big tech ASRs/Speech-to-Text engines like Amazon, Google, IBM and Microsoft.  

The benchmark dataset for this comparison has been a 3rd Party dataset published by an independent party and it includes a wide variety of audio data – audiobooks, youtube videos, podcasts, phone conversations, zoom meetings and more.

Here is a link to some of the benchmarks that we have published.

1.  Link to June 2020 Accuracy Benchmark

2.  Link to Sep 2020 Accuracy Benchmark

3.  Link to June 2021 Accuracy Benchmark

4. Link to Oct 2021 Accuracy Benchmark

5.  Link to June 2022 Accuracy Benchmark

Through this process, we have gained insights into what it takes to deliver high accuracy for a specific use case.

We are now introducing an industry-first relative Speech-to-Text accuracy benchmark to our clients. By "relative", Voicegain’s accuracy (measured by Word Error Rate) shall be compared with a big tech player that the client is comparing us to. Voicegain will provide an SLA that its accuracy vis-à-vis this big tech player will be practically on-par.

We follow the following 4 step process to calculate relative accuracy SLA  

1.  Identify Client Benchmark Dataset

In partnership with the client, Voicegain selects benchmark audio dataset that is representative of the actual data that the client shall process. Usually this is a randomized selection of client audio. We also recommend that clients retain their own independent benchmark dataset which is not shared with Voicegain to validate our results.

2.  Generate golden reference

Voicegain partners with industry leading manual AI labeling companies to generate a 99% human generated accurate transcript of this benchmark dataset. We refer to this as the golden reference.

3.  Run Relative Accuracy comparison

On this benchmark dataset, Voicegain shall provide scripts that enable clients to run a Word Error Rate (WER) comparison between the Voicegain platform and any one of the industry leading ASR providers that the client is comparing us to.

4.  Calculate KPIs for Relative Accuracy SLA

‍Currently Voicegain calculate the following two(2) KPIs 

a. Median Word Error Rate: This is the median WER across all the audio files in the benchmark dataset for both the ASRs‍

b. Fourth Quartile Word Error Rate: After you organize the audio files in the benchmark dataset in increasing order of WER with the Big Tech ASR, we compute and compare the average WER of the fourth quartile for both Voicegain and the Big Tech ASR 

So we contractually guarantee that Voicegain’s accuracy for the above 2 KPIs relative to the other ASR shall be within a threshold that is acceptable to the client. 

How often is this accuracy SLA measured?

Voicegain measures this accuracy SLA twice in the first year of the contract and annually once from the second year onwards.

What happens if Voicegain fails to meet the SLA?

If Voicegain does not meet the terms of the relative accuracy SLA, then we will train the underlying acoustic model to meet the accuracy SLA. We will take on the expenses associated with labeling and training . Voicegain shall guarantee that it shall meet the accuracy SLA within 90 days of the date of measurement.

Take Voicegain for a test drive!

1. Click here for instructions to access our live demo site.

2. If you are building a cool voice app and you are looking to test our APIs, click here to sign up for a developer account  and receive $50 in free credits

3. If you want to take Voicegain as your own AI Transcription Assistant to meetings, click here.

‍

‍

Twilio platform supports encrypted call recordings. Here is Twillo documentation regarding how to setup encryption for the recordings on their platform.

Voicegain platform supports direct intake of encrypted recordings from the Twilio platform.

The overall diagram of how all of the components work together is as follows:

Bellow we describe how to configure a setup that will automatically submit encrypted recordings from Twilio to Voicegain transcription as soon as those recordings are completed.

Configure the Private Key for decryption

Voicegain will require a Private Key in a PKCS#8 format to decrypt Twilio recordings. Twilio documentation describes how to generate a Private Key in that format.

Once you have the key, you need to upload it via Voicegain Web Console to the Context that you will be using for transcription. This can be done via Settings -> API Security -> Auth Configuration. You need to choose Type: Twilio Encrypted Recording.

Configure AWS Lambda function

We will be handling Twilio recording callbacks using an AWS Lambda function, but you can use an equivalent from a different Cloud platform or you can have your own service that handles https callbacks.

A sample AWS Lambda function in Python is available on Voicegain Github: platform/AWS-lambda-for-encrypted-recordings.py at master · voicegain/platform (github.com)

You will need to modify that function before it can be used.

First you need to enter the following parameters:

• voicegainJwt - you need to get the JWT from the same Context that you uploaded the Private Key to
• myAuthConf - this is the name under which you uploaded the Private Key
• expectedPublicKeyId - this is the name under which, on Twilio platform, you uploaded the Public Key

The Lambda function receives the callback from Twilio, parses the relevant info from it, and then submits a request to Voicegain STT API for OFFLINE transcription. If you want, you can modify, in the Lambda function code, the body of the request that will be submitted to Voicegain. For example, the github sample submits the results of transcription to be viewable in the Web Console (Portal), but you will likely want to change that, so that the results are submitted via a Callback to your HTTPS endpoint (there is a comment indicating where the change would need to be made).

You can also make other changes to the body of the request as needed. For the complete spec of the Voicegain Transcribe API see here.

Run a Test

Here is a simple python code that can be used to make an outbound Twilio call which will be recorded and then submitted for transcription.

Notice that:

• We set the URL of the Lambda function in recordingStatusCallback.
• And we tell Twilio to make the callback only when the call recording is completed in recordingStatusCallbackEvent.

‍

It has been over 7 months since we published our last speech recognition accuracy benchmark. Back then the results were as follows (from most accurate to least): Microsoft and Amazon (close 2nd), then Voicegain and Google Enhanced, and then, far behind, IBM Watson and Google Standard.

Since then we have obtained more training data and added additional features to our training process. This resulted in a further increase in the accuracy of our model.

As far as the other recognizers are concerned:

• Microsoft and Amazon both improved, with Microsoft improving a lot on the more difficult files from the benchmark set
• Google has released a new model "latest-long" which is quite a bit better than the previous Google's best Video Enhanced model. Accuracy of Video Enhanced stayed pretty much unchanged.

We have decided to no longer report on Google Standard and IBM Watson accuracy, which were always far behind in accuracy.

Methodology

We have repeated the test using similar methodology as before: used 44 files from the Jason Kincaid data set and 20 files published by rev.ai and removed all files where none of the recognizers could achieve a Word Error Rate (WER) lower than 25%.

This time only one file was that difficult. It was a bad quality phone interview (Byron Smith Interview 111416 - YouTube).

‍

The Results

You can see boxplots with the results above. The chart also reports the average and median Word Error Rate (WER)

All of the recognizers have improved (Google Video Enhanced model stayed much the same but Google now has a new recognizer that is better).

Google latest-long, Voicegain, and Amazon are now very close together, while Microsoft is better by about 1 %.

‍

Best Recognizer

Let's look at the number of files on which each recognizer was the best one.

• Microsoft was best on 35 out of the 63 files
• Amazon was best on 15 files (note that in the October 2021 benchmark Amazon was best on 29 files).
• Voicegain was close behind Amazon by being best on 12 audio files
• Google latest-long was best on 4
• Google Video Enhanced wins a participation trophy by being best on 1 file, which was a very easy "The Art of War by Sun Tzu Full" Librivox Audiobook - WER of 1.79%

Note, the numbers do not add to 63 because there were a few files where two recognizers had identical results (to 2 digits behind comma).

‍

Improvements over time

We now have done the same benchmark 4 times so we can draw charts showing how each of the recognizers has improved over the last 1 year and 9 months. (Note for Google the latest result is from latest-long model, other Google results are from video enhanced.)

You can clearly see that Voicegain and Amazon started quite bit behind Google and Microsoft but have since caught up.

Google seems to have the longest development cycles with very little improvement since Sept. 2021 till very recently. Microsoft, on the other hand, releases an improved recognizer every 6 months. Our improved releases are even more frequent than that.

‍

As you can see the field is very close and you get different results on different files (the average and median do not paint the whole picture). As always, we invite you to review our apps, sign-up and test our accuracy with your  data.

‍

Out-of-the-box accuracy is not everything

When you have to select speech recognition/ASR software, there are other factors beyond out-of-the-box recognition accuracy. These factors are, for example:

• Ability to customize the Acoustic Model - Voicegain model may be trained on your audio data - we have several blogposts describing both research and real use-case model customization. The improvements can vary from several percent on more generic cases, to over 50% to some specific cases, in particular for voicebots.
• Ease of integration - Many Speech-to-Text providers offer limited APIs especially for developers building applications that require interfacing with  telephony or on-premise contact center platforms.
• Price - Voicegain is 60%-75% less expensive compared to other Speech-to-Text/ASR software providers while offering almost comparable accuracy. This makes it affordable to transcribe and analyze speech in large volumes.
• Support for On-Premise/Edge Deployment - The cloud Speech-to-Text service providers offer limited support to deploy their speech-to-text software in client data-centers or on the private clouds of other providers. On the other hand, Voicegain can be installed on any Kubernetes cluster - whether managed by a large cloud provider or by the client.

‍

Take Voicegain for a test drive!

1. Click here for instructions to access our live demo site.

2. If you are building a cool voice app and you are looking to test our APIs, click here to sign up for a developer account  and receive $50 in free credits

3. If you want to take Voicegain as your own AI Transcription Assistant to meetings, click here.

‍

Today, we are really excited to announce the launch of Voicegain Transcribe, an AI based transcription assistant for both in-person and web meetings. With Transcribe, users can focus on their meetings and leave the note taking to us.

Transcribe can also be used to convert streaming and recorded audio from video events, webinars, podcasts and lectures into text.

Voicegain Transcribe is an app accessible from Chrome or Edge Browser and is powered by Voicegain's highly accurate speech recognition platform. Our out-of-the-box accuracy of 89% is on par with the very best. 

‍
Currently there are 3 main ways you can use Voicegain Transcribe:

1. Using browser sharing

If you join meetings directly from your Chrome or Edge browser (without any downloads or plug-ins), then you can use this feature to send audio to Voicegain. Examples of meeting platforms include Google Meet, BlueJeans, Webex and Zoom.

On a Windows device, browser sharing also works with a client desktop app like Zoom and Microsoft Teams. On a Mac/Apple device, browser sharing support desktop apps.

2. App for Zoom Local Recordings

Voicegain offers a downloadable Windows client app that is installed on the user's computer. This app accesses Zoom Local Recordings and automatically uploads them for transcription to Voicegain Transcribe.

Zoom has two types of recordings - Local Recordings and Cloud Recordings. This app is for Local Recordings - where the recording is stored on the hard disk of the user's computer. To learn more about Zoom local recording click here.

Zoom also allows a separate audio file for each participant's recording. Voicegain App supports upload of these individual participant's audio file so that the speaker labels are accurately assigned to the transcript.

‍

3. Upload Audio Recordings

Users may also upload pre-recorded audio files of their meetings, podcasts, calls and generate the transcript. We support over 40 different formats including mp3, mp4, wav, aac and ogg). Voicegain supports speaker diarization - so we can separate speakers even on a single channel audio recording.

‍

Languages Supported

Currently we support English and Spanish. More languages are in our roadmap - German, Portuguese, Hindi.

‍

Advanced Features

Voicegain Transcribe also supports the following advanced Features.‍

a. Projects

Users can organize their meeting recordings and audio files into different projects. A project is like a workspace or a folder.

b. Voice Signatures

Users can save the voice signatures of meeting participants and users so that you can accurately assign speaker labels.

c. Meeting Action Items and Sentiment

Voicegain can also extract meeting action items, positive and negative sentiment.

d. PII Redaction

Users can also mask - in both text and audio - any personally identifiable information.

Coming Soon - Join using meeting url

We are adding a feature where Voicegain Transcribe can join any meeting by having the user just enter the meeting url and inviting Voicegain Transcribe.

We are also adding a Chrome extension that will make it much easier to record and transcribe web meetings.

Get Started for Free today!

By signing up today, you will be signed up on our forever Free Plan - which makes you eligible for 120 mins of Meeting Transcription free every month . Once you are  satisfied with our accuracy and our user experience, you can easily upgrade to Paid Plans.

If you have any questions, please email us at support@voicegain.ai

‍