HMM "sp" has unsupported arc

[kmaclean]

When starting Julius in interactive mode, this error is appears:

Warning: HMM "sp" has unsupported arc. SERIOUS WARNING: Transition arc has been modified as below SERIOUS WARNING: This may cause unintended recognition result

Seems to have no noticeable effect on recognition.

[kmaclean]

E-Mail sent to Julius Support (LEE Akinobu (ri@…))

I am trying to compile an English Language Acoustic Model using the HTK (r3.3) toolkit, but keep getting the following error when I run Julian (r3.5):

###### build up system
Reading in HMM definition...(ascii)...Warning: initial state has more than one arc
Warning: HMM "sp" has unsupported arc.
SERIOUS WARNING: Transition arc has been modified as below
SERIOUS WARNING: This may cause unintended recognition result
 0.000000e+00 1.000000e+00 0.000000e+00
 0.000000e+00 9.161721e-01 8.382791e-02
 0.000000e+00 0.000000e+00 0.000000e+00
limit check passed
   defined HMMs:    50
  logical names:   506 in HMMList
    base phones:    44 used in logical
done

Note: even though I get this error on startup, it does not seem to affect my recognition rates.

I use the following procedure to create my sp model:

create new "sp" model in hmm4/hmmdefs as follows:

• copy and paste the "sil" model and rename the new one "sp"
• remove state 2 and 4 from new "sp" model (i.e. keep 'centre state' of old "sil" model in new "sp" model)
• change <NUMSTATES> to 3
• change <STATE> to 2
• change <TRANSP> to 3
• change matrix in <TRANSP> to 3 by 3 array
• change numbers in matrix as follows:

         0.0 0.7 0.3
         0.0 0.6 0.4
         0.0 0.0 0.0

I'm not sure what the transition matrix should look like in order to correct the error.

Any help would be greatly appreciated.

thank you,

Ken

[kmaclean]

Hi,

> Warning: HMM "sp" has unsupported arc.
> SERIOUS WARNING: Transition arc has been modified as below
> SERIOUS WARNING: This may cause unintended recognition result
>  0.000000e+00 1.000000e+00 0.000000e+00
>  0.000000e+00 9.161721e-01 8.382791e-02
>  0.000000e+00 0.000000e+00 0.000000e+00
> limit check passed
>    defined HMMs:    50
>   logical names:   506 in HMMList
>     base phones:    44 used in logical
> done

>         ┌───────────────────────────────┐
>         │ 0.0 0.7 0.3                                                  │
>         │ 0.0 0.6 0.4                                                  │
>         │ 0.0 0.0 0.0                                                  │
>         └───────────────────────────────┘

Normal version of Julius/Julian? assumes that all the transition matrix in hmmdefs have only one arc from initial state and only one arc to final state, and should not have direct transition from initial state to final state in a model. This is a known limitation of normal version of Julian.

> Note: even though I get this error on startup, it does not seem to affect my
> recognition rates.

When Julius/Julian? find such model, it tries to modify the transition not to have such transitions. The "SERIOUS WARNING" mentions about that. The results may be almost the same, but since the modified transition of "sp" does not allow whole skipping of the "sp" model, the result may not be strictly the same as intented at training time and may cause some error.

There is an another version of Julius/Julian? called "multi-path version" that enables arbitrary trantision (the decoding will become slower instead). I recommend trying this. The version can be compiled with configure option "--enable-multipath".

Please note that, even with the "multi-path version", Julius does not allow a definition of a word in which all phone HMM have such "whole model-skip transition", because it requires word-level skipping and hard to handle at decoding process. For example, if you have "sp" model with model-skipping transition, you can attach or insert the sp model to any position of other words in the dictionary, but you can not define a word with only the "sp" model. In such case, Julius/Julian? outputs error.

[...]

Thanks, --

LEE Akinobu (ri@…) Department of Computer Science and Engineering Graduate School of Engineering Nagoya Institute of Technology Gokiso-cho, Showa-Ku, Nagoya, 466-8555 Japan Tel. +81-52-735-7913 --- (Edited on 6/14/2006 11:04 pm [GMT-0400] by kmaclean) ---

[kmaclean]

> Just to confirm that I understand what you are saying, the sp model
> only has 3 states.  The transition matrix describes all the
> transition combinations for those 3 states.  So on the first row of
> my transition matrix, the first column gives the transition from
> state 1 to state 1, the next column gives the transition from state
> 1 to state 2, and the last column the transition from state 1 to
> state 3 (the final state in this HMM model), and so one for the
> other two rows.
>
> So basically I should not have an entry in the third column of the
> first row, because that is mapping a transition from the initial
> state to the final state - and Julian does not permit this (though
> the multi-path version of Julian does).

Yes, that's right. ",1] ); //-->

> Is there somewhere in the HTK book that describes what values I should be
> putting in the transition matrix for the "sp" model?  I struggled to
> understand the HTK tutorial on this point a while back, and basically had to
> review the mailing list archives for hints on what should go into the
> transition matrix.

Please note that the limitation of transition matrix is only for Julius/Julian?, and not for HTK. The HTK can treat any transition.

How you should put transition matrix is an issue of acoustic modeling. If you want to model occasional short-pause insertions between words that often occur in English, a typical solution may be to define a sp model with "model-skippable transition", i.e. transition from initial state to final state, and add it to the end of every words. It expresses that the short-pause to be able to appear between words. LEE Akinobu (ri@…)

Department of Computer Science and Engineering Graduate School of Engineering Nagoya Institute of Technology

[kmaclean]

Will try the following transition matrix:

         ┌───────────────────────────────┐
         │ 0.0 1.0 0.0                   │
         │ 0.0 0.9 0.1                   │
         │ 0.0 0.0 0.0                   │
         └───────────────────────────────┘

so we don't have to recompile Julius with configure option "--enable-multipath"...

[kmaclean]

Acoustic Model Creation

How-to:

Already updated ... Compile_AcousticModel.sh script, make_hmm4 procedure:

make_hmm4 () {
    LINE="start"
    NUM=1

	rm -rf ./interim_files/hmm4/hmmdefs.tmp1
	rm -rf ./interim_files/hmm4/hmmdefs.tmp2
	
    while [ `echo ${LINE} | awk '{ print $1 }'` != "" ];
    do
		LINE=`cat -n ./interim_files/hmm4/hmmdefs | grep ^[[:space:]]*${NUM}[[:space:]] | sed s/^[[:space:]]*[[:digit:]]*//g`
		if [ `echo ${LINE} | awk '{ print $2 }'` = "\"sil\"" ]; then
            while [ `echo ${LINE} | awk '{ print $1 }'` != "<ENDHMM>" ];
            do
                echo ${LINE} >> ./interim_files/hmm4/hmmdefs.tmp1
                echo ${LINE} >> ./interim_files/hmm4/hmmdefs.tmp2
				print_process ${NUM}
				let "NUM += 1"
				LINE=`cat -n ./interim_files/hmm4/hmmdefs | grep ^[[:space:]]*${NUM}[[:space:]] | sed s/^[[:space:]]*[[:digit:]]*//g`
			done
			
			echo ${LINE} >> ./interim_files/hmm4/hmmdefs.tmp1

			NUM2=1
			while [ ${NUM2} != "28" ];
			do
				LINE2=`cat -n ./interim_files/hmm4/hmmdefs.tmp2 | grep ^[[:space:]]*${NUM2}[[:space:]] \
				| sed s/^[[:space:]]*[[:digit:]]*//g`

			   case ${NUM2} in
					1 ) 
						echo ${LINE2} | sed s/~h\ \"sil\"/~h\ \"sp\"/g >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
					2 ) 
						echo ${LINE2} >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
					3 ) 
						echo ${LINE2} | sed s/5/3/g >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
					10 ) 
						echo ${LINE2} | sed s/3/2/g >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
					11 | 12 | 13 | 14 | 15)
						echo ${LINE2} >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
					22 )
						echo ${LINE2} | sed s/5/3/g >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
					24 ) 
						echo "0.000000e+000 1.000000e+000 0.000000e+000" >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
					25 ) 
						echo "0.000000e+000 0.900000e+000 0.100000e+000" >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
					26 ) 
						echo "0.000000e+000 0.000000e+000 0.000000e+000" >> ./interim_files/hmm4/hmmdefs.tmp1
						;;
				esac
				let "NUM2 += 1"
			done
        fi
		echo ${LINE} >> ./interim_files/hmm4/hmmdefs.tmp1
		print_process ${NUM}
		let "NUM += 1"
	done
	mv -f ./interim_files/hmm4/hmmdefs.tmp1 ./interim_files/hmm4/hmmdefs
	return 0
} 

Tutorial

Step 7 of Tutorial updated to include the following:

First copy the contents of the hmm3 folder to hmm4.  Then using an editor, create new "sp" model in hmm4/hmmdefs as follows:

    * copy and paste the “sil” model and rename the new one “sp”
    * remove state 2 and 4 from new “sp” model (i.e. keep 'centre state' of old “sil” model in new “sp” model)
    * change <NUMSTATES> to 3
    * change <STATE> to 2
    * change <TRANSP> to 3
    * change matrix in <TRANSP> to 3 by 3 array
    * change numbers in matrix as follows:

         0.0 1.0 0.0
         0.0 0.9 0.1
         0.0 0.0 0.0

Your sp model should look something like this:

[kmaclean]

From Julius Manual:

Multi-path version --- extended support for AM To make decoding faster, the default Julius has several limitations in the HMM state transition: it allows only one transition from start node and to end node within a model.

Thus, it does not support a multi-path model that has several phoneme models in parallel, or a skip transition that has direct transition from start node to end node to allow skipping of all the output state.

You can compile a Julius with "--enable-multipath" to overcome these limitations and allow arbitrary transitions. The compiled one is called "multi-path" version, and it can handle ANY arbitrary transition, but the decoding speed is slightly slower than the normal version.

[kmaclean]

Updated web site to use Julian-3.5.2-multipath