Princess Diana wasn’t a fan of Scotland. She hated the midges, disliked the inclement weather, and counted down the days until she could return home to Kensington Palace.

Fortunately, the Duchess of Sussex is much more enthusiastic about the Highlands, which is just as well; Meghan Markle and Prince Harry may soon be spending more time in the country, and specifically at the Castle of Mey, the coastal estate that’s been in the royal family since 1952.

After spending nearly a week of their summer holiday at Mey and a brief weekend at Balmoral with the Queen and the Duke of Edinburgh, the Sussexes have fallen in love with the country, much to the delight of the Queen and Prince Charles, who is hoping Harry and Meghan may want to get even more involved with the castle in the future. “I think Charles might want Harry, if he is interested, to oversee the operations of Mey when the time is right,” reveals a source.

Beginning in January, the newly created Prince’s Foundation, which brings together several of Prince Charles’s charities, will become the sole corporate trustee of the Castle of Mey. It is currently run by the Queen Elizabeth Castle of Mey Trust, which was set up in 1996 by the Queen Mother before her death in 2002. The Queen Mother acquired the castle and grounds in 1952, not long after her husband, King George VI, died, and spent holidays there every year until her death. Charles has retained close links with the castle and is currently president of the Queen Elizabeth Castle of Mey Trust.

“I think Prince Charles would probably be delighted if his younger son were to take an interest in Mey,” said Charles’s biographer Sally Bedell Smith. “What I don’t know is when Charles would want, if indeed he does, to hand the supervisory reins to Harry. My guess would be when Charles becomes king and his official duties as monarch multiply.”

Meghan and Harry spent almost a week with Charles this summer at the castle, which is situated on Scotland’s north coast and has a walled garden and acres of beautiful parkland. Charles, who adored his late grandmother, loves the property, which is open to the public from May to September.

Charles rents the castle for several days every August and is often joined by Camilla. “Camilla doesn’t stay too long, she loves her bucket-and-spade holidays in Devon with her grandchildren, so it must have been a thrill for Charles to have Meghan and Harry stay,” says the source. “I’m not even certain the Duchess of Cornwall has stayed for such a long stretch.”

The private holiday puts to rest rumors that Charles is not getting along with his sons, which inspired a rare statement of denial from the Palace. A family member adds to Vanity Fair that the prince is “closer to Harry than ever,” adding that he is forging a “lovely relationship with the duchess.”

“The wedding was a real turning point,” the family member continues. “Charles showed so much love towards Harry and Meghan, and I think Harry will be eternally grateful to his papa for stepping in and giving Meghan’s hand in marriage. The bond between them is very close.”

According to the source, the trio spent evenings dining at the castle, reading, walking dogs, and talking. “Charles has been very supportive of Meghan and everything she is going through with her own father. He’s got a real soft spot for her and thinks she’s the best thing to have happened to Harry.”