You can remember the meaning of 所谓 with this easy mnemonic:
The so-called restaurant is just a place (所) people speak of (谓) when they want drugs.
Using mnemonics to remember the meaning of Chinese characters like can have an enormous impact on how long to learn Mandarin it takes.