It’s going to cause a problem but it gets a bit more complicated than that. The facility has to be in use for a diplomatic or consular purpose. You can easily think of obvious examples where this doesn’t work. If the PRC constructed a small airfield next to its mission in say, Vancouver with the permission of the Canadians, and then started flying drone sorties from it against Seattle and proclaimed it was an inviolable facility that wouldn’t work as a defense. Even if Canada agreed, you can’t use a consulate as a military base to launch attacks.
This is a grayer area of course, but revolutionary states in the Middle East have a history of playing in this gray zone. Plenty of terrorist attacks have been planned and launched from diplomatic facilities. The Libyans famously shot a protester in the middle of London from their mission. The aforementioned BA bombings were planned and directed inside of the Iranian embassy in Argentina.
You generally lose inviolability if a facility is used to break laws. The normal response is expulsion but in this case the facility is operating with permission of the Syrians. The presence of so many IRGC operatives raises the obvious question of what the facility really was— they weren’t doing visas or promoting commercial ties.