They were always the signal that the train was about to go through the crossing and they have become American icons. The caboose on a train served a purpose at one time, but now have largely disappeared from mainline railroads.
In the simplest terms, a caboose was the last car of the train, where the conductor and brakemen rode. The brakeman was responsible for making sure the brakes worked properly, for applying the brakes on each car, making sure the couplings were secure, and watching for problems throughout the train, such as a load shifting, or track problems. The train crew didn't need to stay with the passengers, but needed separate quarters so their comings and goings would not disturb the passengers. On a freight train, the caboose provided the only sleeping quarters on the train. It began as a shed nailed to a flat car, but eventually, an actual railroad car was manufactured that was safer and more suited to its purpose.
As evidenced in the line from the song, "Little red caboose, chug, chug, chug," the car was usually painted red, often for visibility. It also provided a window either on top or to the sides, so the brakeman had a better view of what was going on with the train. Different kinds of cabooses were manufactured to meet various railroading needs, but they all served essentially the same purpose. The origin of the word name, however, is a mystery. Some say it is a version of the Dutch word kabhuis, for the main sailing compartment on a ship. Some say it comes from the French camboose which has essentially the same meaning as the Dutch word. In any event, "caboose" has entered the language, both as the word for its actual object, and in American slang, as a word for a woman's derriere.
Carrying a caboose was law on U.S. railroads until the 1980s and the introduction of the Flashing Rear End Device (FRED). The FRED negated the need for a the final red car, since its blinking red light serves as a warning for a train ahead, it sends signals to the engineer that the couplings are sound and the load is distributed properly. Since it was no longer needed and only added weight to the train, the car began to disappear.
A train with a caboose is a rarity nowadays. Sometimes they are included on special trains, when nostalgia is the aim of the excursion. They also may serve as crew quarters on repair trains. Still, when watching a train go by, it just doesn't seem complete without the funny little caboose bringing up the rear.