Use of TTC system in LHCb

The TTC system has been developed as a part of the RD12 project to distribute all the required synchronization (clock and resets) and trigger signals to the front-end systems of LHC experiments. A large set of documentation for the TTC system and its building block are available on the web (TTC system, TTCrx, TTCvi, TTCrx, etc.).  Two types of broadcast facilities are used in LHCb to send LHCb specific reset signals, event readout types and event destination addresses to the front-end electronics. The control of the TTC system in LHCb is performed by the Timing and Fast Control system (TFC).

The detailed use of the TTC system in LHCb is described in several LHCb notes. (se under general front-end and TFC)

The used of reserved TTC commands and bit fields by specific sub-detectors can only be allowed after an official acceptance by the LHCb electronics coordination !.

Clock

The TTC system can distribute the 40 MHz bunch crossing clock to a large number of receivers with relatively small jitter. The clock phase at the receiver end is determined by delays in the distribution system. The different TTC receivers (TTCrx) will receive the clock with different clock phase as it is not intended/required to equalize the delays in the distribution network. Each TTCrx has two sets of high resolution (0.1ns) delay generators which can be used to precisely align the local clock to the detector signals or align interface signals between modules.

Warning: Be aware that TTCrx clock in many cases can NOT be used to drive high speed serial links directly because of its jitter. A special jitter filter chip called QPLL has been designed in the CERN micro electronics group to decrease the jitter sufficiently to drive the GOL serializer.

Bunch structure synchronization and identification

The bunch structure identification is in the TTC system based on a local 12 bit bunch ID counter (B-ID). This counter is aligned with the LHC machine by the regular issue of a bunch ID reset synchronized to the beginning of the LHC bunch structure (end of large bunch gap). The exact phase alignment of this to the LHC, taking into account delays in the TTC network, is performed in two steps. The phase alignment with-in a clock period can be taken care of by one of the clock delay generators in the TTCrx. The alignment at the level of complete clock periods is taken care of by a programmable pipeline delay in the TTCrx of up to 15 clock cycles. The bunch count reset is issued to the front-end via a clock synchronous 8 bit broadcast command with bit<0> being the direct bunch ID reset signal.

The generation of local bunch identifiers in the front-end is supported by two mechanisms. The TTCrx generates an external correctly aligned bunch count reset signal which can be used by local bunch id counters in the front-end. In addition it has an internal Bunch id counter which value can be available on an output bus for each L0 trigger accept (can alternatively be used for event ID).

The bunch count ID generated in the front-end controlled by the TTC system is related to the bunch ID of events at the output of the L0 pipeline. The bunch ID related to the input of the L0 pipeline is of little interest in most cases as all L0 buffers in the front-end are of constant latency. The Bunch ID at the input of the L0 buffer can be calculated from this by adding the L0 pipeline delay taking into account that the bunch ID does not return (overflow) to zero after 4095 but after 3563.

L0 trigger

The L0 trigger decision (called L1 in all other LHC experiments) is transmitted on a separate 40 MHz clock synchronous channel . Nearly any sequence of L0 triggers can be distributed by the TTC system (Max 24 consecutive accepts). The received L0 trigger is phase aligned with the local clock from the TTCrx delay generators and can in addition be delayed by up to 15 additional clock cycles (same as used for bunch count reset). A clock synchronous accept/reject signal is finally generated on an output pin of the TTCrx. For each L0 accept an internal L0 event ID counter in the TTCrx is incremented.

Event identification

In addition to the bunch crossing identification of each accepted event the TTCrx supports the generation of a 12/24 bit L0 event ID (L0-ID). This event ID counter is incremented for each L0 trigger accept and initialized to zero by the event count reset. An event count reset is generated by the issue of a short broadcast command with the event count reset signal directly mapped into bit<1> of the command data.

A multiplexed output bus is available to generate the B-ID and the L0-ID of accepted events. To observe both the B-ID and the L0-ID two clock cycles are needed on the multiplexed bus following a L0 accept. As LHCb requires the acceptance of consecutive triggers the multiplexed bunch ID/ event ID scheme can NOT be used. The TTCrx has an alternative working mode where the output bus is used only for bunch ID or event ID. It is recommended to use the Bunch ID mode and generate the event ID by external counters ( in ASICs , FPGA's, etc.).

Reset of L0 front-end electronics

An encoded short broadcast command is used to send a reset command to the front-end electronics. Three individual reset bits are available in this reset command. After LHCb has abandoned the L1 trigger, and now uses a direct 1MHz readout to the DAQ system, only one of these reset bits are actively used (L0 front-end reset)

Read-Out Type (ROT)

An encoded short broadcast command is used to distribute to the L1 front-end electronics the Read-Out Type (ROT) of each event to allow specific processing to be performed on different types of events.

MEP event destination

Multi Event Packet (MEP) from the L1 front-end electronics to the DAQ system are sent to specific CPUs in the DAQ farm controlled by a 12bit destination address sent by a long TTC broadcast. The number of events to put in each MEP is given by the number of preceding ROT broadcasts.

Broadcast command

Short broadcast:

The short 8 bit broadcast command is used to distribute several important signals to the front-end. This broadcast has been implemented especially for this kind of purpose and is protected against single bit errors by the use of hamming code error detection/correction. The allocation of the 8 available bits are the following:

Generic:
<0>    Bunch count reset (B-ID reset)
<1>    Event count reset (L0-ID reset)
These have been hardwired in the TTCrx into specific bits of the TTC short broadcast.

LHCb specific:
<7:2>  LHCb command

Long broadcast:

The long broadcast is a special version of individually addressed commands with a TTCrx address of zero which selects all TTCrx receivers (making it a broadcast). The 8bit data and 8 bit local address can be considered as a 16bit wide broadcast message.

Bits

15

14

13

12

11

10

09

08

07

06

05

04

03

02

01

00

Command

Brd. type

MEP destination

1

0

L0-ID[1:0]

MEP destination address[11:0]

CMD5 (reserved)

0

0

x

x

x

x

x

x

x

x

x

x

x

x

x

x

CMD6 (reserved)

0

1

x

x

x

x

x

x

x

x

x

x

x

x

x

x

CMD7 (reserved)

1

1

x

x

x

x

x

x

x

x

x

x

x

x

x

x

Use of broadcasts in L0 front-end electronics

For the use of the TTCrx chip in the L0 front-end electronics, only a limited number of signals from the broadcast commands needs to be used. Long broadcasts must be simply ignored. The B-ID Reset and the E-ID reset signals (short broadcast bits 0 and 1) can be used directly as synchronous reset signals to counters as the broadcast strobe is included in these (see TTCrx manual). For the remaining short broadcast bits ( 7: 2 ) a validation with the broadcast strobe 1 ( BrcstSt1) is required. A synchronous L0 front-end reset can be generated with the following product term: (Brcst[7] = 0) and (Brcst[6] = 1) and (Brcst[2] = 1) and ( BrcstSt1 = 1). In a similar fashion the default calibration pulse type can be generated with: (Brcst[7] = 0) and (Brcst[6] = 0) and (Brcst[5] = 0) and (Brcst[4] = 1) and (Brcst[3] = 0) and (Brcst[2] = 0) and ( BrcstSt1 = 1).

Individually addressed TTC commands

The TTC receiver supports the use of individually addressed commands. THIS IS NOT SUPPORTED IN LHCb.

TTC receiver

All sub-detectors must use the TTCrx receiver chip. Several sub-detectors have chosen to use the TTCrq plug-in card with the optical receiver, TTCrx and a QPLL for clock jitter filtering.

Be aware that in case the JTAG port of the TTCrx is unused the TCK and TRST signals must be connected to passive signal states to prevent the JTAG port to interfere with the correct function of the TTCrx (no pull-up resistors on these signals as normally defined in the JTAG standard).

It is brought to the attention of TTCrx users that it has been observed that the I2C interface of the TTCrx chip can only be expected to work correctly when the chip is connected to the TTC fiber with a valid TTC clock

Overview of the number of TTCrx chips, optical receivers and TTCrqs ordered and distributed to the different sub-detectors

TTC optical receiver

The TTCrx have been characterized to work with an optical receiver from Truelight and one from Agilent. As the Truelight receiver have the best performance and best radiation resistance this is the chosen receiver for LHCb use. The Truelight receiver has unfortunately reached its end of normal product availability and a large quantity of these receivers have therefore been purchased to cover the total LHCb needs. How to connect this to the TTCrx can be found on the schematics of the TTCrq plug-in card.