StateTransition's are used when a state's end scope is recognized. At that point the state is forced into a different state (usually a different language) specified in the StateTransition. An example is a "<script>" tag. The StateTransition occurs at the ">" and JScript is entered.
The pattern group you define within the StateTransition is what is used to match within the child language. If that pattern is matched, the language exits and the text is rescanned using the parent language.