Nadira Povey

Search This Blog

Showing posts with the label LMs

View all

Posts

Reconstruct the Original Training Data from N-Grams.

on November 04, 2021

LMs

+

0

Get link
Facebook
X
Pinterest
Email
Other Apps

What is HCLG.fst?

Trying to understand what is needed to make HCLG.fst file.

ASR model and HCLG.fst LibriSpeech

Many final models final.mdl in various folders, which one is the final one we need?

End-to-end generally refers to things like CTC, transformers, RNN-T. Generally, anything where you can code the entire thing with PyTorch/TensorFlow without C++ code. but again, End to end is not really a meaningful or important category. Generally it's considered trendy to be end to end.

Reconstruct the Original Training Data from N-Grams.

"$train_cmd"

Dynamic Data Parallel

The normal way in Pytorch to use multiple GPUs is to use DDP (Dynamic Data Parallel). In Pytorch documentation they have a very good tutorials. There is a certain boiler plate code we should use. We have to invoke in a special way. It will produce several versions of the process and each version will use a different GPU. It will not change the code, it will be only wrapper around the code. Just by having a code base we normally can't say if it is using multiple GPUs. But we can search for words "rank", "world_size" and "ddp". Please look at this project to get an idea about how one might use DDP https://github.com/k2-fsa/icefall/search?q=ddp

LibriSpeech training

LibriSpeech training Step 0: Make a copy of run.sh Step 1: Install flac Step 2: Change cmd.sh Step 3: Needed if only one GPU is used Step 4: Create a screen to run the script. Step 5: Run run_edited.sh LibriSpeech training will need at least one GPU and 400GB storage space. The tutorial below was created using AWS instance. Step 0: Make a copy of run.sh Normally I copy the run.sh to new file that I can edit. But we can edit original run.sh if wanted. cp run.sh run_edited.sh Need to edit path to where we want to put all downloaded folders nano run_edited.sh Edit the new version. Add few lines after line 7. Pick a data folder location that you want. I chose the one below. mkdir -p nadira/data data=nadira/data Line 17 was data=/export/a15/vpanayotov/data , If you don’t have that folder you can create it, or just make a data folder. run_edited.sh top few lines: #!/usr/bin/env bash # Set this to somewhere where you want to put your data, or where # someone else has already put i...

Install Kaldi: Ubuntu

Install Kaldi: Ubuntu Step 1: Choose an instance on AWS Step 2: Download Kaldi from GitHub Step 3: Read the Kaldi installation instructions and follow Option 1 Step 3: Go to tools/ and follow INSTALL instructions there. Step 4: go to src/ and follow INSTALL instructions there. Step 1: Choose an instance on AWS Select the g4dn.2xlarge instance as it has 8 vCPUs and 1 GPU, and its 75 cents per hour, so make sure to pause the instance when you’re not using it. Need 10 days of training. For Step 3, select a specific subnet. This is optional since it defaults to a random region. For Step 4, disable the “Delete on Termination” setting and change the Root’s size to 600 GB. From personal experience, 400GB isn’t going to be enough. Then, you can skip to step 7, review the instance settings, and launch the instance. I’m creating a key pair but you can use an existing one. Step 2: Download Kaldi from GitHub git clone https://github.com/kaldi-asr/kaldi.git kaldi --origin upstream cd kaldi git p...

Tested train_lm.sh with --include-heldout

In Kaldi we can create LMs using script train_lm.sh. When building a language using this script one can use a flag --include-heldout. If the the flag is not used kaldi withhelds some data for collecting stats about the data and uses the rest to build LM. If we have very little data we must use this flag. Tested train_lm.sh with --include-heldout

Install Kaldi: Red Hat

Install Kaldi: Red Hat Step1: Install git on AWS Instance Step 2: Download Kaldi from GitHub Step 3: Read the Kaldi installation instructions and follow Option 1 Step 4: go to tools/ and follow INSTALL instructions there. Step 5: go to src/ and follow INSTALL instructions there. How to Install Kaldi Red Hat| Tutorial written tutorialhttps://npovey.github.io/jekyll/update/2022/01/01/Install-Kaldi-Red-Hat.html https://www.youtube.com/watch?v=YR_kcw6FOd0&ab_channel=NadiraPovey I’ll be using an AWS EC2 instance for the installation but you can do this directly on a linux machine. RedHat Step1: Install git on AWS Instance (Keep in mind that the yum install is only for red hat, for debian/ubuntu it would be apt-get install.) https://cloudaffaire.com/how-to-install-git-in-aws-ec2-instance/ sudo yum update -y sudo yum install git -y Step 2: Download Kaldi from GitHub https://kaldi-asr.org/doc/install.html git clone https://github.com/kaldi-asr/kaldi.git kaldi --origin upstream cd kaldi...

Powered by Blogger