Once he had his languages, Tolkien sought to map out their historical relationships, beginning (philologist as he was) with a prototypical speech he at first called "Gnomish" (the Noldor were known as the "Gnomes" in his very early works). He then used what he knew of the principles of language change both to create and explain the relationship between this ancient Elvish, Quenya, and its more "worn down" relative "Sindarin." In doing so, he used well-known principles of historical language study, which suggested (for instance) that /v/ might "wear down" into /b/ and /k/ into /g/. Thus the Quenya "Valaröke" (spirit of fire) "wore down" into the Sindarin "Balrog." Eventually, he ran into issues which could only be explained by modifying the history of the Elves to explain the changes (or lack thereof). He knew that a population's speech, once separated from its main body, would begin to evolve differently, and thus difference required separation -- the Light Elves from various Dark Elves, the High from the Low, the Elves of Middle-Earth from those of Beleriand, and so forth.
The language of Men was always meant to have had a separate origin, though greatly influenced by the contact of the Edain with the Eldar. The more "noble" men spoke a speech with more Elvish influence, and borrowed many words from them. Early men spoke a language known as "Hadorian," which was the parent language of Adûnaic, the speech of Númenor. With the fall of that realm, the few Faithful brought their speech back into Middle-Earth, where it seems to have grown into a sort of "prestige dialect" of Gondor; the bulk of the rest of men spoke a distant tongue, which by late in the Third Age had become known as "Westron." Later in his career, Tolkien liked the conceit of having merely "translated" the works of Bilbo and Frodo out of Westron; he then rationalized the use of Old English to represent the speech of Rohan because the Rohirrim spoke an older form of Westron.
Not many examples, alas, of Westron, alas, are known; Tolkien principally talks about it in the Appendices to LOTR. The names of the Hobbits, for instance, are known; Frodo's actual name was Maura (the -a was a masculine ending in Westron names); Merry (Meriadoc) was Kalimac, with "Merry" representing the more familiar "Kali," and so forth.
The study of Tolkien's languages is now a field in and of itself, and has advanced greatly since Ruth S. Noel's little red book (which is now known to contain many errors). Linguists have had the advantage of access Tolkien's unpublished writings, including some which never even made it in the twelve volumes of the History of Middle-Earth. Some of their work is quite technical! But the fascination with invented speech is a key part of both Tolkien's and his fans' engagement, and the idea that it was speech -- language -- from which his world was wrought, provides a remarkable starting-point for the analysis of the worlds which resulted.
Note: The image at the top of this post is of a letter from Aragorn, after he had become king, announcing a visit to the edge of the Shire early in the Fourth Age. Tolkien made many facsimiles of this letter; this one I made myself back in 1979 after having copied one of these versions from his original manuscripts.