Drumry is first mentioned in the 1328 Exchequer rolls when the Lady of Drumry paid the Chamberlain of Scotland one chalder of flour at the feast of St. Martin for the freedom of her lands. It seems likely that this was either Elena Livingston, who was married to Andrew Livingston; or their daughter-in-law, Margaret, who married their son William and are mentioned later as lairds in 1338. William was to die in 1339. The next laird, their son William, was to marry Christian of Callander and inherited the barony of Callander in by his father-in-law's forfeiture in 1345.

With their estates in West Lothian, the Livingstons were a powerful family. In 1302, Andrew's brother, Sir Archibald Livingston was listed as only one of two Scots noblemen who supported Edward Longshanks, the English king, in his claim to Scotland; the other being the Earl of Dunbar. Later that year, Robert the Bruce was also to feign loyalty to Edward – but while Robert was later to challenge Edward after the death of his father in 1304, when he had a stronger claim to the throne – the Lothian held Livingston lands continued to be a vital lifeline for Edward as he tried to keep Stirling Castle out of Bruce’s hands. Bruce was made King in 1306. Edward Longshanks died in 1307 and the battle against King Robert I was taken up by the new King of England, Edward II. Bruce took Linlithgow when his men were hidden under hay to penetrate the guard. Around 1308-9, both Rutherglen and Dumbarton fell to Bruce and his allies, and given its proximity, the estate of Drumry would follow suit.

Given Bruce's victory at Bannockburn in 1314, the Livingston family switched allegiance to Bruce. Indeed, William Livingston, grandson of Andrew who was a supporter of Edward Longshanks, became a comissioner to deal with the ransom of Bruce's son King David II in 1357. The next laird Sir Robert would further increase the Livingston estate by marrying a daughter of Sir Michael Wemyss and taking the lands of East Wemyss. (The Wemyss family were staunch supporters of Bruce in the Wars of Independence; Michael Wemyss had been petitioned by John Weston for supporting Bruce in 1306 and Edward Longshanks had forfeited his land in Midlothian. David Wemyss signed the Declaration of Arbroath in 1320.)

Sir Robert was just one head of the Livingston clan. Sir John Livingston was now living in Callander, but he was to die in the Battle of Homildon Hill in 1402. He left a son, Alexander, who was later to become an advisor to King James II.

In the early 15th century, there is the first mention of St. Mary's Chapel in Drumry. At this time Robert Livingston, as the laird of Drumry, was patron of the church. He also granted lands in Fife to the Abbot of Dunfermline in 1437. The fortunes of the Livingstons were to rise with Sir Alexander being regent to the young king but in 1449, with the regent away in England, James II, arrested many of the prominent Livingstons and later executed two - one of whom was Robert Livingston of Drumry.