jorme,
From the timing of your posts yesterday, I'm guessing that you got the converter installed and charged the batteries for a couple of hours before your test in which the voltage started dropping. If that's the case, then you didn't have enough time to fully charge the batteries (even though with no load they read 6VDC each). It will take about 24 hours for your converter to top off the batteries. After that, it should read 13VDC +/_ a little for at least 30 - 45 minutes after turning on 2 or 3 lights. You "might" have bad batteries, but maybe not. If you test with 12 VDC deep cycle batteries and it works, make sure your wiring to the batteries is connected correctly. It's pretty easy to "confuse" the black/red vs black/white and wind up blowing another converter if the fuses don't protect your new investment !!!!!
Keep us posted on what you find
John
ADDED: OOPS I just saw your most recent post about the batteries being fully charged, so, if they really are fully charged, then you "might" have a problem with the 6 VDC batteries. Possibly, only one of them is not holding a charge, but to find out which, you'd best put them on a load tester at an auto store. AUTOZONE will do it for free. Once you know the status of those batteries, you should be in business, either with new 6VDC or 12VDC batteries