Correlation Processing

Correlation processing is divided into two steps as follows.

Fringe Search

Correlation processing for geodetic VLBI data is usually carried out with a small number of lags such as 32 to increase the throughput of processing. To keep a correlation peak at the center of the lag window, clock offset should be determined with an accuracy of better than a few tenths percentage of a sampling period. Fringe search is a correlation processing to find out fringes and to determine a clock offset between two stations.

A-priori delay neccesary for correlation processing is calculated from time of ovservation, station positions, source position, earth orientation parameters, and clock error. It is difficult to obtain an accurate clock error between two stations in advance, so that a clock error measured by GPS at each station is usually used as a temporal value to calculate a-priori delay. If there is no GPS clock measurement, zero is used as a temporal vaule. In case of geodetic VLBI lasting for 24 hours or more, clock rate is slso determined.

The concrete method of fringe search and clock parameters determination is out as follows.

1. Determination of the scan for fringe search

Source information is found in a schedule file. It is also obtained by executing "apri_calc" with a schedule monitoring mode as follows. 
	apri_calc -/home/vlbi/sked/sample.skd 

	   or

	apri_calc /home/vlbi/sked/sample.skd -monit

	   where "/home/vlbi/sked/sample.skd" is a schedule file
then observation information is displayed as follows. 
	apri_calc (Ver. 2016-10-12)

	====================== RUN CONDITION ========================
	  K5 file naming type is  Type 1 : sidDDDNNNN.dat (SKED deflt)
	=============================================================
	SkdMonit: *********** Schedule file monitor **********
	SkdMonit: Schedule file = ipvlbi/sked/sample.skd
	SkdMonit: Expcode       = KS07235
	SkdMonit: Total Scan #  = 593
	SkdMonit: Total Star #  = 16
	SkdMonit:
	SkdMonit:  1st   Scan   = 2007/08/23 01:15:00  3C84
	SkdMonit:  Last  Scan   = 2007/08/24 00:46:20  3C273B
	SkdMonit:
	SkdMonit: ---------- Station ID Table ----------
	SkdMonit:  G --- KOGANEI
	SkdMonit:  R --- KASHIM11
	SkdMonit:  Y --- TATEYAMA
	SkdMonit: --------------------------------------
	SkdMonit: ------- Star Table -------------------
	SkdMonit:          NAME1      NAME2  R.A.(deg)   DEC(deg)   EPOCH
	SkdMonit:   1   0059+581          $  15.690677  58.403093 2000.000000
	SkdMonit:   2   0316+413       3C84  49.950667  41.511695 2000.000000
	SkdMonit:   3   0420-014          $  65.815836  -1.342518 2000.000000
	SkdMonit:   4   0552+398          $  88.878357  39.813657 2000.000000
	SkdMonit:   5   0727-115          $ 112.579635 -11.686833 2000.000000
	SkdMonit:   6   0923+392    4C39.25 141.762558  39.039126 2000.000000
	SkdMonit:   7   1226+023     3C273B 187.277915   2.052389 2000.000000
	SkdMonit:   8   1253-055      3C279 194.046527  -5.789312 2000.000000
	SkdMonit:   9   1308+326          $ 197.619433  32.345495 2000.000000
	SkdMonit:  10   1334-127          $ 204.415762 -12.956859 2000.000000
	SkdMonit:  11   1641+399      3C345 250.745042  39.810276 2000.000000
	SkdMonit:  12   1730-130    NRAO530 263.261274 -13.080430 2000.000000
	SkdMonit:  13   1921-293          $ 291.212733 -29.241700 2000.000000
	SkdMonit:  14   2134+004    2134+00 324.160776   0.698393 2000.000000
	SkdMonit:  15   2145+067          $ 327.022744   6.960723 2000.000000
	SkdMonit:  16   2251+158    3C454.3 343.490616  16.148211 2000.000000
	SkdMonit: --------------------------------------
	SkdMonit: ------- Frequency (MHz) Table --------
	SkdMonit: Gr# 1    7714.99 U   7724.99 U   7754.99 U   7814.99 U
	SkdMonit: Gr# 2    8034.99 U   8234.99 U   8414.99 U   8524.99 U
	SkdMonit: Gr# 3    8564.99 U   8584.99 U   2154.99 U   2164.99 U
	SkdMonit: Gr# 4    2234.99 U   2294.99 U   2384.99 U   2414.99 U
	SkdMonit: --------------------------------------
	SkdMonit: ------- PCAL Freq (kHz) Table ---------
	SkdMonit: Gr# 1      10.0     10.0     10.0     10.0
	SkdMonit: Gr# 2      10.0     10.0     10.0     10.0
	SkdMonit: Gr# 3      10.0     10.0     10.0     10.0
	SkdMonit: Gr# 4      10.0     10.0     10.0     10.0
	SkdMonit: --------------------------------------
Then select a strong source such as "3C273B", and execute "apri_calc" with a schedule monitoring mode with the option "-source" as follows. 
	apri_calc -/home/vlbi/sked/sample.skd -source 3C273B

	  or

	apri_calc /home/vlbi/sked/sample.skd -monit -source 3C273B
A certain baseline can be specified if necessary by using the "-baseid" option as follows 
	apri_calc /home/vlbi/sked/sample.skd -monit -source 3C273B -baseid RY
where "RY" is a combination of Station IDs. Then scan information is displayed as follows. 
	SkdMonit: --------- PICKUP SCAN TABLE ----------
	SkdMonit: SCAN#     SOURCE YYYY/DDD HH:MM:SS DURA  STATION_IDS
	SkdMonit:    10     3C273B 2007/235 01:36:03   62  G R Y
	SkdMonit:    18     3C273B 2007/235 01:50:59   62  G R Y

	   .............................


	SkdMonit:   571     3C273B 2007/235 23:51:47   62  G R Y
	SkdMonit:   579     3C273B 2007/236 00:10:11   62  G R Y
	SkdMonit:   586     3C273B 2007/236 00:29:04   62  G R Y
	SkdMonit:   593     3C273B 2007/236 00:46:20   62  G R Y
	SkdMonit: --------------------------------------
Column named "SCAN#" shows the scan number.

2. Calculation of a-priori values

Execute "apri_calc" for a scan selected by the previous step. 

 
	apri_calc /home/vlbi/sked/sample.skd -start 10 -stop 10 -apedir /home/vlbi/corrapri
where "-apedir" option sets the directory of a-priori files created. If you want to execute for all scans selected, execute as follows. 
 
	apri_calc /home/vlbi/sked/sample.skd -source 3C273B -baseid RY -apedir /home/vlbi/corrapri
See "How to run apri_calc" for details.

3. Correlation Processing

Execute "fx_cor"ior "fx_cor_new" for not K5/VSSP format data) with a large number of lags such as 1024 or more as follows. 

fx_cor /home/vlbi/corrapri/apesample.txt -lag 1024
where "apesample.txt" is a a-priori file created by "apri_calc". After processing, correlation amplitude and residual delay are displayed by channel as follows. 
	  ==============================================================
	     CH#    FREQ(MHz)      MAX AMP        RESIDUAL DELAY (sec)
	  --------------------------------------------------------------
	      1      8209.99      0.000651            -3.26108e-06
	      2      8219.99      0.000443            -3.28622e-06
	      3      8249.99      0.000589            -3.23678e-06
	      4      8309.99      0.000480            -3.25464e-06
	  ==============================================================

	COUT File is ../cout/cout0005.txt
where "COUT File" is a correlation results file. Correlation functions are also displayed in a separate window as follows.


Figure 1. Correlation functions displayed after correlation processing.

If clear peak appears in the plots of correlation function, we can determine clock offset. If it is not clear, execute the coarse-search program "sdelay" to search a clear peak as follows. 

	sdelay ../cout/cout0005.txt
where "../cout/cout0005.txt" is COUT File (correlator out file) displayed at the end of "fx_cor" processing. After "sdelay" processing, summary of coarse search is displayed as follows and search function is plotted (Figure 2). 
	 ******************** SDELAY (Ver. 2016-08-12) SUMMARY OUT PUT ************
	   COUT     : ../cout/cout0005.txt
	   X DATA   : /home/kondo/data/testspeed/Xk5data.10.dat
	   Y DATA   : /home/kondo/data/testspeed/Yk5data.10.dat
	   BASELINE : KASHIM11 - TOMAKO11
	   SOURCE   :     3C273B           SAMPLING  : 1 bit   8 MHz
	   PRT      : 2003/197 02:41:10    Tinteg(s) : 9.0
	   LAG SIZE : 1024
	   CLOCK    : offset    5.735e-06(s)   rate    0.000e+00(s/s)
	   EOP      : ut1-utc 0.000000(s)
	            : x-wobb  0.000000(asec)
	            : y-wobb  0.000000(asec)

	  ========================================================================
	   CH#  FREQUENCY       AMP MAX     POSITION         RESIDUAL
	           (MHz)                  (2048x 128)  Delay(usec) Rate(ps/s)  SNR
	  ------------------------------------------------------------------------
	     1    8209.99 U    1.192e-03  ( 973,  65)     -3.251    -0.134    10.1  <=check correlation amp and residuals
	     2    8219.99 U    8.613e-04  ( 973,  68)     -3.270     2.482     7.3
	     3    8249.99 U    1.089e-03  ( 973,  65)     -3.239    -0.089     9.2
	     4    8309.99 U    1.095e-03  ( 973,  66)     -3.242     0.943     9.3
	  ------------------------------------------------------------------------
	   Note: No amplitude correction is made.
	  ========================================================================

	  ======================= PCAL SUMMARY ============================
	   CH#   PCAL FREQ(kHz)     X-Amp  X-Phase    Y-Amp  Y-Phase
	  -----------------------------------------------------------------
	    1         10.00         0.109    70.6     0.251   111.9
	    2         10.00         0.107   -92.2     0.244  -162.7
	    3         10.00         0.106   136.2     0.247    84.3
	    4         10.00         0.104  -143.5     0.259   -43.2
	 *************************************************************************

	Outfile is ./sdelayout.txt

Figure 2. Search functions displayed after "sdelay" processing. Peaks are not clear due to a large number of lags. In this case, zoom up using options "-tzoom" and "-tshift". Figure 3 shows a search function the options "-tzoom 20 -tshift -3.25e-6".
Figure 3. Same as Figure 2 but with the options "-tzoom 20 -tshift -3.25e-6".

Fringe detection is judged from summary output and plot of search functions as follows.

If above conditions are almost satisfied, we can say "fringe is successfuly detected".

4. Determination of clock offset

Clock offset coffset is given as


where co is clock offset used at "apri_calc" and Δτ is residual delay obtained by "sdelay".

5. Determination of clock rate

Clock rate crate is given as


where c1 is clock offset obtained for a scan at the time t1 and c2 is clock offset obtained for a scan at the time t2, and t1 and t2 are PRT (processing reference time) given by a unit of second. If a session extends on the next day, take the day difference into consideration.

6. Set up clock parameters at "apri_calc"

Clock offset, clock rate, and clock epock are set up when executing "apri_calc" as follows.

Processing All Scans

1. Calculation of a-priori values

Calculate a-priori values for all scans with reflecting clock parameters obtained by fringe search. An example of non-interactive execution is as follows. 

	apri_calc  /home/vlbi/sked/jd1606.skd -coffset -3.25e-6 -crate 1.0e-13 -cepoch 2016/197-02:41:10
	           -baseid RY -g 2 -xdir /home/vlbi/data/R  -ydir /home/vlbi/Y -apedir /home/vlbi/corrapri
Conditions used in this example are as follows; clock offset is -3.25 μsec, clock rate is 1.0x10-13(s/s), clock epock is 2016/197 02:41:10, baseline is "RY", data directory for X station data is "/home/vlbi/data/R" and for Y station "/home/vlbi/data/Y", directory for a-priori files is "/home/vlbi/corrapri". In case of using clock offset only, omit options "-crate" and "-cepoch".

2. Generation of the list of a-priori files 

	ls -1 /home/vlbi/corappri/ape*RYb.txt > apelistRY.txt
where the directory of a-priori files is "/home/vlbi/corrapri" and "apelistRY.txt" is a file generated. "ape*RYb.txt" is a filter to select baseline "RY" and frequency code "b" if there are a-priori files for multiple baselines. If you want to select all a-priori files, execute like, 
	ls -1 /home/vlbi/corappri/ape*.txt > apelist.txt
where "apelist.txt" can be any another name.

3. Correlation processing for all scans As for 1-bit sampling data, execute as follows, 

	cor_all ./apelistRY.txt
where "apelistRY.txt" is the name of list file located in the current directory. See "How to run software correlator" for other options. As for multi-bit sampling data, such as 2-bit AD data, use "fx_cor_all" instead. Use "cor_all_new" or "fx_cor_all_new" for format data other than K5/VSSP format.

Correlation Processing for a Variety of Format Data (VDIF, Mark5B, ADS, OCTAD)

When executing "apri_calc", data format should be specified using options "-format" or "-formX", "-formY". (See "How to execute apri_calc" for details.) Then use "cor_new", "fx_cor_new", "cor_all_new", "fx_cor_all_new" instead of "cor", "fx_cor", "cor_all", "fx_cor_all".

updated on February 23, 2017