← All projectsCOSC 10 / Software

Part-of-speech tagging

Tagging words in a sentence with a hidden Markov model.

Training the model

The tagger reads tagged sentences into two maps: transition scores between parts of speech and observation scores from tags to words. It converts counts to log probabilities before decoding, so a sentence score is the sum of transition and observation scores.

Dynamic programming and backtracking

For each word, decodeViterbi keeps the best score reaching each next tag. A backtrace map records the preceding tag that achieved that score. After the final word, the algorithm selects the highest-scoring state and follows those stored choices backward to recover the tag sequence.

Course test runs

The supplied test harness first checks two hard-coded graphs, then evaluates the file-based simple and Brown datasets. In this capture, both hard-coded predictions match their expected tag sequences; the test harness reports 86.49% accuracy on the simple set and 96.47% on the Brown set.

Mithun RameshkumarDartmouth College