In order to handle these, the Xcfe desktop environment uses a plugin (xfce4-xkb) that has some problems in that it sometime cannot remember settings after reboot or logout.
My solution is this:
1. Go to Setting Manager - Keyboard - Layout, and set the languages in the preferred order (delete them and add them in order to achieve this: in my case, first the default, English-US, French, Romanian standard). Go to the properties of the xkb plugin and make the same settings in the same order with the same default language.
2. Open the file
gksudo gedit /home/your-username/.config/xfce4/xfconf/xfce-perchannel-xml/keyboard-layout.xml
Edit it to look similar to this.
3. Open the file
Edit it to look similar to this:
(paste from here)
4. Open the file (root)
gksudo gedit etc/default/keyboard
Edit it accordingly:
Possibly the last is not strictly necessary to make it work, but just in case...
Also, as I use LXDE (Lubuntu) from time to time, before doing the above in Xubuntu/Xfce, I made these settings in Lubuntu/LXDE.