Synchronization

Home Electronics E-mail Notes Meetings Subsystems Search

Synchronization of front-end electronics

The whole front-end electronics system must be perfectly synchronized to the bunch collisions of the LHC machine to capture correctly the signals from the different detector elements. In addition all the L0 pipeline buffers located on thousands of modules must be correctly synchronized at the clock level with the L0 trigger system to insure the correct extraction of event data accepted by the L0 trigger. A complicated system with many different delays must be aligned to each other, using timing parameters extracted during special calibration runs of the system.

Phase alignment to bunch collisions

The phase of the clock used to sample the detector signal at the input of the L0 pipeline buffer must be precisely aligned with the bunch collisions in the interaction point taking into account the particle flight time and the delay in the detector and the analog front-end. In most cases the sampling of the detector signal must be performed at the peak of the signals induced by the traversing particles. Because of the high bunch crossing rate of LHC, and the requirement to limit pile-up and spill-over to a minimum, the signals generated by the analog front-end normally have peaking times (and pulse width) shorter than the 25ns bunch crossing interval. The sampling time must therefore be controlled with high precision and stability (sub nano second). The correct phase alignment can only be obtained during special calibration runs where the clock phase is adjusted until the correct capture of real detector signals are obtained.

In some cases the clock phase alignment can be performed for a group of channels together. This though requires a careful matching of delays between the channels in the group ( flight time, detector delay, delay in analog front-end, cable delays, sampling delay, clock distribution on module or chip, etc.).

The TTCrx is capable of generating two independent clocks which phase can be controlled with 0.1ns resolution. Normally only one TTCrx IC is mounted on a front-end module thereby in most cases not supplying sufficient number of programmable clocks. Different clock phases can also be generated with commercial delay circuits, or long chain of gates in a FPGA or ASIC can be used to generate non calibrated delays ( in many cases the calibration procedure can take this into account). 

When the front-end signals have been captured with the correct phase it is in most cases required to synchronize the data to one common clock for the whole front-end module to enable all L0 pipelines on a module to run synchronously. This can be performed quite straight forward if the phase spread between channels on a module is below one clock cycle (should be the case for all modules in LHCb). The clock used to drive the L0 pipeline can (should) be delivered from a TTCrx as the synchronous L0 trigger decision also comes from this source.

Synchronization to LHC bunch structure

After phase alignment of the individual detector signals and synchronization to a common module clock all data in the front-end are available in a sampled format at the rate of the LHC bunch crossings. The alignment with the LHC bunch structure (in steps of clocks) must be performed to assign correct bunch crossing identification to the acquired data. The bunch identification is later used in many parts of the system to correctly align data from different sources and also perform extensive synchronization checks of data while the system is running.

In most cases the assignment of correct bunch crossing ID to event data is not performed before an event is accepted by the L0 trigger (see later). This reduces significantly the amount of data to store in the L0 pipeline buffer and can be used because the L0 is a constant latency trigger. In the case a sub-detector needs to deliver data to the L0 trigger processor system the assignment of correct bunch identification to data from a module needs to be performed before they are correlated with data from other front-end modules (see later).

The TTC system has been designed to support the assignment of correct bunch count ID in the front-end electronics. A dedicated bunch count reset is distributed to all TTCrx receiver chips using a broadcast command. The actual bunch count reset can be delayed by up to 16 clock cycles before it is actually used to reset an internal bunch counter in the TTCrx and generate an external reset signal to control external bunch id counters. The programmable delay of 16 clock cycles ( 400ns) should be sufficient to cover the skews between different front-end modules in LHCb. In the TTCrx the bunch ID is only available at its output when a L0 trigger accept have been generated and is considered to be the bunch ID of the data accepted by the L0 trigger. The correct bunch ID of data at the input to the L0 pipeline can be calculated from this by adding the L0 latency (160) taking into account that the Bunch ID does not return to zero after 4095 (normal binary roll-over) but at 3563 (LHC machine cycle length = 3564).

Synchronization to L0 trigger system

The synchronization of data from different front-end modules to the L0 trigger system is a non trivial task. Data coming from different parts of the front-end system may have different clock phase and have different pipeline delays. Several different schemes can be used to re-synchronize data in the trigger system:

  • Assign bunch ID to all data and re-align based on this
  • Realign based on programmable delays of data from different sources
  • Use the sequence of data from each source starting from a first data-set being marked after a reset .

After processing detector data in different L0 trigger subsystems the results must be merged and realigned before the final trigger decision can be taken. At this point significant skews between different trigger subsystems may have accumulated and it is advisable to tag all data at this level with a bunch ID. The overhead of carrying the bunch ID tags together with the local trigger results are at this point minimal (only few links required to carry trigger information from local trigger processors to global L0 trigger decision unit).

The final trigger decision must be delivered to the readout supervisor with a constant latency as defined in EDMS 566457.

Synchronization to L0 trigger decision

The final part of the synchronization of the whole L0 front-end electronics is the extraction of accepted events from the L0 pipeline. It is assumed that the clock used to write into the L0 pipeline is the same as used to accept/reject data at the output of the pipeline thereby not needing any clock phase alignment at this point. Different parts of the front-end will have different delays in the arrival of data into the pipeline and will also receive the L0 trigger decisions with different delays caused by different cable (optical) delays from the trigger supervisor. The same programmable delay of up to 15 clock cycles in the TTCrx, as used for the bunch ID reset, is intended to compensate for this.

Synchronization verification on L1 front-end electronics

All event data received by the L1 front-end electronics must be verified to come from a L0 front-end branch that is correctly synchronized. Bunch ID and Event ID information in the event data must be verified against reference information from local TTCrx receiver chips on the L1 front-end itself.