Justin Bieberand Hailey Baldwin are very much enjoying being newlyweds — so much so that the couple is finally publicly embracing their new relationship status.

“After being married for a few weeks, they both realized that they actually felt like husband and wife without having a religious ceremony,” a source tells PEOPLE about Bieber, 24, and Baldwin, 22, whosecretly wedin September, just two months aftergetting engaged.

“They want to openly celebrate their marriage,” the source says. “This is why Hailey decided to change her name. She loves being married to Justin. They are still planning a wedding and a religious ceremony.”

The model recentlychanged her Instagram handleto “Hailey Bieber” and even showed off a newdiamond necklaceof her husband’s surname. And days before changing her name on social media, she also wore a custom Levi’s trucker jacket with hernew married namesplashed across the back at the opening of the brand’s new Times Square Flagship store in N.Y.C.

Hailey Baldwin and Justin Bieber.Kahlea Baldwin/Instagram

baldwin-3

In addition, the Biebers also celebrated their first Thanksgiving together as a married duo, as well as her Nov. 22 birthday in his native country.

“They had a small birthday celebration for Hailey with his family and spent the rest of the weekend relaxing in Canada,” the source says. “Justin loves being in Canada because it’s very peaceful. He can take Hailey out and it’s not a mess with photographers. They are much more left alone. They seem very happy together.”

Bieber’s mom, Pattie Mallette, also shared happy family photos and videos from their celebrations, including footage of her sonsmashing a huge sheet cake into his wife’s face.

Justin Bieber and Hailey Baldwin.Mark Blinch/NHLI via Getty Images

justin-bieber

During the holiday weekend, the pop star reflected on how much it meant to him to becelebrating his first Thanksgiving as a married man.

“First Thanksgiving as a married man, first time hosting Thanksgiving. First time both sides of the family has come together. Relationships are hard and love isn’t always easy but thank you Jesus for showing me how,” Bieber captioned aphotoof a snowy forest on Instagram. “Every day is a learning process, trying to be more like him, patient, kind, selfless, boy I got a long way! But gods grace is sufficient!”

source: people.com