Using multiple SI5351 outputs

My home build VNA uses a single SI5351 to generate both the test signal and the LO for the downmix to the audio IF where the I/Q demodulation is done.
The VNA has some problems in isolation and one possible cause is leakage from one output of the SI5351 to the other output so we should measure how good/bad this is.
One output of the SI5351 is connected to the SA with 30dB attenuation. The other output of the SI5351 is not connected. The third output is always mute and not frequency has been setup
The SW steering the SI5351 uses for each output a separate VCO to have fully independent frequency control.
Different attenuation levels where used to verify the seen signals where not created due to IMD in the SA.
First with 10MHz on one outputs and the others muted. Seems OK, all spurs are at least 70dB down.
And yes, there is a small mistake in the auto marker algorithm of the SA




Then with a second output active at 15MHz. The second output is visible at -55dB, but also a strong spur at 5 MHz (-62dB). This already explains the isolation problems in the VNA




Will it get worse at higher frequencies? First with two outputs both set to 50MHz, a nice clean signal



And now with the connected output set to 50MHz and the open output to 52MHz. The level of the spurs is only 42dB down. When you send this as LO to a mixer with a LO-IF isolation of 30dB the not connected output will appear at -72dB in the IF output of the mixer which fully explains the (lack of) isolation performance of the VNA using this SI5351 



But is this a result of the output stages doing some mixing or is it earlier? Lets test with the connected output again set to 10MHz and and the open output to 66MHz



Measurement at other frequencies have confirmed that the direct output leakage increases with increasing frequency but the other spurs (like seen above) are not directly related to the selected output frequency. The modulo  difference between the frequencies of the two outputs determines the level and position of the spurs and these follow a pattern that repeats a number of times over the total frequency range. It may be possible that the coupling/leakage is through the PLL loop in the SI5351. More to investigate.