I am not sure why but I suspect that the now-silent h in this word is a really modern change to English, and that not so very long ago hour was pronounced exactly the same way as it is still written today: [ˈhaʊə(ɹ)].

This does not appear to be the case. The word hour is ultimately from Latin hora, but it came into English through Anglo-Norman French, and (as tchrist said in a comment) we have evidence that the word lost the original [h] sound long before it ever entered English. The loss of Latin [h] occurred in all Romance languages, suggesting this loss was shared and therefore occurred early on. Furthermore, we have early spellings in Romance languages that lack the letter H. Any cases of the word being pronounced with /h/ in English must have originally been spelling pronunciations. It is very possible for a spelling pronunciation to spread and become established as the standard pronunciation of a word, but this does not seem to have ever happened for hour.

had the system of writing a for the indefinite article before a vowel sound and writing an for the indefinite article before a consonant sound not yet become standard during the 1700s?

This system had already developed centuries before then, although some small details of the system have changed over time (some details remain variable even today). The works of Shakespeare, created more than 200 years earlier, show the two forms a and an distributed based on whether the following sound is a consonant or a vowel.

The search tool at OpenSourceShakespeare shows a clear contrast between heart and hour, suggesting that in Shakespeare's time, as today, the first started with the consonant /h/ and the second started with a vowel:

• 39 total hits for "a heart" vs 0 total hits for "an heart"
• 0 total hits for "a hour" vs. 82 total hits for "an hour"

Caveat: I haven't consulted original documents of Shakespeare's work to check these results. Open Source Shakespeare is based on Moby Shakespeare, which normalizes the spellings of Shakespeare's words based on modern spelling. I'm not certain whether the use of a vs. an was affected by normalization according to the modern a vs. an rule; however, my guess is that it was not, as you can see from that search tool that Shakespeare's uses of "an hundred" (10 total hits there), which is nonstandard today, evidently have not just been normalized away to "a hundred".

Or is this just an incorrect and repeated typo in that page from The Literary Gazette from the 18th century?

It seems most probable that it is a typo, or if not, something similarly erratic. Note that you can find "an" used on the same page in "the run of an Hour-glass" and in "the time of an hour", so even though "a Hour-glass" appears more than once, it does not appear consistently in that text in place of "an Hour-glass". Therefore, the occurrence of three examples of "a Hour-glass" is puzzling, but not very clear evidence that "Hour-glass" was actually pronounced with /h/ by the author of that text.