What Is TrackBack?
In a nutshell, TrackBack was designed to provide a method of notification between websites: it is a method of person A saying to person B, "This is something you may be interested in." To do that, person A sends a TrackBack ping to person B.
TrackBack ping: a ping in this context means a small message sent from one webserver to another.
And why would person B be interested in what person A has to say?
Person A has written a post on his own weblog that comments on a post in Person B's weblog. This is a form of remote comments--rather than posting the comment directly on Person B's weblog, Person A posts it on his own weblog, then sends a TrackBack ping to notify Person B.
Person A has written a post on a topic that a group of people are interested in. This is a form of content aggregation--by sending a TrackBack ping to a central server, visitors can read all posts about that topic. For example, imagine a site which collects weblog posts about Justin Timberlake. Anyone interested in reading about JT could look at this site to keep updated on what other webloggers were saying about his new album, a photo shoot in a magazine, etc.
Source: http://www.movabletype.org/trackback/beginners/
Further Reading
Moveable Type Development Blog
Movable Type Trackback Tutorial
How TrackBack Works by Michael d'Cruftbox
SamRuby's article on why TrackBack's are useful
See also: WikiTrackBack