Implementation of mean-timing and subsequent logic functions on an FPGA ∗ J. Bieling, G. Ahluwalia, J. Barth, F. Klein, J. Pretz 2 Physikalisches Institut, Universit¨at Bonn, 53115 Bonn, Germany 1 0 2 H. Fischer, F. Herrmann, C. Schill, S. Schopferer n a Physikalisches Institut, Universit¨at Freiburg, 79104 Freiburg, Germany J 8 1 ] t Abstract e d - This article describes the implementation of a mean-timer and coincidence logic on s n a Virtex-5 FPGA for trigger purposes in a particle physics experiment. The novel i feature is that the mean-timing and the coincidence logic are not synchronized with . s c a clock which allows for a higher resolution of approximately 400 ps, not limited by i a clock frequency. s y h p Key words: FPGA, mean-timing, coincidence logic, trigger, asynchronous, [ unclocked logic 2 v 5 3 7 4 9. 1 Introduction 0 1 1 : For scattering experiments in particle physics one often needs fast trigger v i decisions. The trigger signal should have a good time resolution with respect X to the occurrence of the scattering event, of the order of 1 ns. Scintillation r a detectors allow for fast trigger decisions with an appropriate time resolution. However, in large scintillation detectors the time resolution is deteriorated by a varying light propagation time depending on the impact point of a particle inthedetector.Thus,suchlongscintillatorstripsarereadoutonbothsidesby photo multiplier tubes (PMTs), and the method of mean-timing is employed by generating in real time a signal whose time corresponds to the average time ∗ corresponding author Email address: [email protected] (J. Pretz). Preprint submitted to Elsevier 19 January 2012 of the individual PMT pulses and is thus independent of the impact point. Various methods have been developed to meet this requirement [1,2,3]. In this article an implementation of a mean-timer logic and a subsequent coincidence logic is presented. The goal was to use an already existing multi- purposeFPGAboardequippedwithaXilinxVirtex-5allowingforamaximum clock frequency of 500 MHz. The corresponding granularity of 2 ns was not enough for our application, so we decided to use an unclocked logic which does not underlie the discreteness of the clock cycles. Up to now such a mean-timer was only implemented on a CPLD [4]. The paper is organized as follows. Section 2 describes the requirements of the system and the FPGA board used. The main Section 3 presents the im- plementation of the mean-timer and coincidence logic on the FPGA board. Section 4 discusses the implementation and performance of the system at the COMPASS experiment at CERN [5]. 2 Requirements of the system and the FPGA board 2.1 The COMPASS Trigger System Fig. 1 explains the main features of the trigger system at the COMPASS experiment [6]. Muons produce light in two scintillator hodoscopes placed along the beam direction. The scintillator strips are read out on both sides by PMTs.Afterconversiontologicsignals,thesearepassedontothemean-timer. For offline analysis, the single PMT signals are digitized by time-to-digital- converters (TDCs). In a subsequent coincidence logic, the mean-timer outputs of the two ho- doscopes are passed on to a coincidence matrix which allows to select coin- cidences between the strips of the two hodoscopes. By selecting only certain channel combinations of H1 and H2, as indicated in Fig. 1, it is possible to discriminate scattered particles from beam halo particles. Both hodoscopes have 32 strips, so in total 64 mean-timers and a 32×32 coincidence matrix are needed. Since the length of each scintillator strip is 250 cm, which corresponds to a light propagation time of approximately 20 ns, the dynamic range of each mean-timer is required to be at least ±20 ns. 2 mean timer mean timer scattered muon target x halo muon o mean timer mean timer y z x 0m 6m 16m Fig.1.Triggerprinciple.Particlesaredetectedintwoscintillatorhodoscopesplaced at different positions along the beam axis. The 250 cm long scintillator strips are read out on both sides with photo multiplier tubes (PMTs). After conversion to a logical signal, using discriminators, the mean-timing is done. The output signals of the mean-timers are sent to a coincidence matrix unit. Selection of the coincidences (grey squares) allows to trigger on particles originating from the target region and to suppress background events originating from beam halo particles. 2.2 The FPGA Board For the implementation of the mean-timer and the coincidence logic, we use a custom made FPGA board developed at the University of Freiburg [7]. This so called GANDALF board can be equipped with two input cards providing 2 × 64 LVDS1 inputs and in addition 2 NIM2 inputs and 4 NIM outputs (see Fig. 2). The board is constructed as a 6U-VME64x/VXS module and can be controlled via the VME-Bus. It is based on a Xilinx Virtex-5 (XC5VSX95T- 2) FPGA [8]. This FPGA consists of an array of 160×46 configurable logic blocks(CLB).EachoftheseCLBscontains8programmablelogicelementsand variousotherelementslikeacarrylogicandflip-flops.Theprogrammablelogic elements are called look-up tables (LUTs) and have 6 inputs and 2 outputs, so that any 6-to-2 logic can be implemented. To allow communication between different LUTs and flip-flops, every CLB has a switch matrix to which all elements of a CLB are connected. If two elements reside in different CLBs their switch matrices use so called global interconnects. The FPGA provides 1 Low Voltage Differential Signal 2 Nuclear Instrumentation Module 3 NIMput CFloamshpact 1 In Memory VME64x S CPLD based Interface to 64 LVDInputs BoVarMdE C Ionntefirgfuarcaet,ion inVsMidEe- CCPraUte 2 NIMOutputs Xilinx Virtex-5 XC5VSX95T-2 NIMput 1 In S VDuts Memory 64 LInp Controller 2 NIM Outputs 1Q4D4RMIbI D4DGRb2 Fig. 2. Simplified layout of the GANDALF-Board. 640 I/O connections, from which 256 are used for the 128 LVDS inputs, and 6 for the NIM inputs and outputs. To each of these connections belongs a dedicated delay element which allows to delay the signal in 64 steps of 75 ps. Thus the maximum delay is about 5 ns. A 36 kbit block-Ram, a dynamic clock manager (DCM) and some CLBs optimized for digital signal processing algorithms complete the structure of the FPGA. Due to variations in the production process, each FPGA has its own timing behavior. To be able to design clocked circuits, however, some maximum prop- agation timings must not be exceeded. Therefore, Xilinx characterizes every singleFPGAbyitstimingbehaviorbysocalledspeedgradesrangingfrom1to 3. A speed grade of 1 corresponds to the slowest version [9]. In the simulation software the developer has to select the correct speed grade. It is indicated by the last digit in the device number, in our case an FPGA with speed grade 2 is used. 4 output logical OR logical ANDs delay taps Δ Δ ML MR t=t* Δ Δ SL SR L R t=t t=t0 t=t l r PMT scintillating strip y 250 cm x Fig. 3. Principle of the mean-timer using a tapped delay line. 3 Implementation of Mean-Timer and Coincidence Logic 3.1 Implementation of Mean-Timers Themean-timersareconstructedusingthetappeddelayline(TDL)method[1] as shown in Fig. 3 and App. A. The two input signals of each mean-timer propagate in opposite directions in the two delay lines and trigger the ANDs between these lines while passing each other. The OR-ed output of all ANDs provides the desired mean-timed signal. Implementing an unclocked mean- timer on a FPGA, one has to take care that (1) all delays on the tapped delay line are of equal amount (up to a few ps) and (2) the delays from the ANDs to the logical OR are identical. A chain of LUTs is used to build the tapped delay line. It turns out that the delays between the 8 LUTs inside the same CLB are much smaller than 5 the delays between two LUTs located in two different CLBs, because the second case requires global interconnects. Since more than 8 delay elements are needed to reach the desired dynamic range, the tapped delay lines have to be constructed using those interconnects. To determine the path length of a particular interconnect between two LUTs, we used LOC-Constraints (location constraints) to place the LUTs at certain positions and added a signal path from one of the two outputs from the first LUT to one of the 6 input pins of the second LUT. Furthermore we tagged that path with a MAXDELAY constraint of a few ps to enforce the shortest path between these two LUTs. After the Xilinx Software finished the place and route process, we extracted the simulated path length from the logfile. To be sure that exactly this path will be used in the later design, we used the Xilinx FPGA-Editor to extract the ROUTE-Constraint of the simulated path. This process had been automated to simulate all conceivable combinations and we found two oppositely running paths between adjacent CLBs with an equal length of 579 ps. However, these paths lengths depend on the absolute position of the used adjacent CLBs in the CLB grid of the FPGA, but it was possible to identify enough CLBs with the same simulated timing behavior to construct 64 identical mean-timers. The resulting design is illustrated in Fig. 4. The propagating signals in the TDLs are not only sent to the next delay element but also to one of the inputs of the ANDs. These ANDs are realized in one of the 6 remaining LUTs of each CLB (2 are already used for the tapped delay lines). The propagation time from the TDL LUTs to the ANDs has to be identical only within one mean-timer, differences between different mean-timers can be compensated by delays of each input. To implement 64 mean-timers in 46 columns with 160 CLBs each, two mean- timers need to be placed in one column. We choose to design the tapped delay lines with 53 delays and thus 54 LUTs which amounts to a total range of 53×579 ps ≈ 30 ns. The mean-timers are placed at the top and at the bottom of 32 CLB columns. This is advantageous for the implementation of the coincidence logic between the two hodoscopes discussed in Section 3.2. The last step consists of implementing the OR of all 54 AND outputs for each mean-timer. As stated before, each LUT has only six inputs, thus the OR has to be realized as a cascade of multiple ORs. All signals of the same level of this cascade should have equal delays. The search for suited combinations has been performed with an automated script, as for the TDLs. To be able to place all 64 mean-timers side by side, the paths of the cascade are not allowed to leave the CLB column. A solution was found using a 6 level cascade with 2 to 1 ORs where the propagation time variation is at most 34 ps. These solutions are different for every mean-timer, leading to different total propagation times which are again compensated by the input delays of each mean-timer. 6 ↓ & ↑ CLB ↓ & ↑ CLB ↓ & ↑↑↑ CLB Fig. 4. Routing of the tapped delay lines and coincidences inside the FPGA for one mean-timer. The background picture was generated by the ISE software tool [10]. 7 s 0.5 n 0.4 1 / 0.3 /ns +ttrl - m2 00..021 +tr2 0.8 t --00..21 tl - m --00..43 -0.5 t 0.6 6.5 7 7.5 8 8.5 9 9.5 10 t-t /ns l r 0.4 0.2 0 -0.2 -0.4 -25 -20 -15 -10 -5 0 5 10 15 20 25 t-t /ns l r Fig. 5. The difference of the mean-time generated by the FPGA, t , and the calcu- m lated mean-time (t +t )/2. l r Fig. 5 shows the result of a test measurement where two input signals derived from two slightly detuned clocks are sent to the two inputs of one mean- timer and simultaneously to TDCs for time measurement. The x axis shows the time difference of the input signals, t − t . The y axis shows the time l r differencebetweentheoutputsignalofthemean-timer,t ,andthemean-time m calculated from (t +t )/2. The propagation time of the signals through the l r FPGA including the coincidence logic described in Sec. 3.2 is approximately 70 ns. The output shows the desired behavior in a time range of ±20 ns, i.e. the mean-time is independent of the time difference t − t . One also clearly l r observes a modulation in the inset which shows a magnification on the region 6.5 to 10 ns of the x-axis, due to the tapped delay line. The observed jitter of the order of 120 ps (RMS) as seen in the projection on the y-axis in Fig. 6 is acceptable. The observed slope of 5 ps per ns originates from a slightly different propagation time in the two TDLs which was not present in the simulation. At the borders the mean-time signal comes later by up to 200 ps. The reason for this effect is the following. If one of the signals already left the tapped delay line when the second signal just enters, the AND is not triggered by the leading edge of the signal leaving the tapped delay line, the mean-time signal is correspondingly delayed. For the data shown in Fig. 5 this effect was minimized by shortening the input signals inside the FPGA to a width of 1 ns. 8 s Mean 0.001229 e12000 ri RMS 0.1118 t n e f o10000 b. n 8000 6000 4000 2000 0 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 t+t t - l r /ns m 2 Fig. 6. Projection of Fig. 5 on the y-axis. The total dynamic range of ±20 ns is shorter than predicted by simulations (±30 ns). This is due to the simulation software which overestimates the prop- agation time. This software is designed for clocked applications for which an upper limit of propagation time must be guaranteed, however, most signal paths are faster than simulated. The influence of temperature variation of the FPGA on the propagation time of the signals through all logic elements inside the FPGA can be neglected. It was measured to be 30 ps/K in a range from 40 to 80◦C inside the FPGA chip. 3.2 Implementation of the coincidence matrix logic The coincidence logic has to be implemented in a way that arbitrary combina- tions between 32 inputs from the first hodoscope (H1) with 32 inputs from the second hodoscopes (H2), i.e. in total 1024 ANDs are selectable in a so called coincidence matrix. One of the 1024 matrix elements is shown in Fig. 7. The selection is done by using a third input on a AND gate which is set to a logic 1 if the channel combination is selected. The trigger signal is then given by an OR of all 1024 ANDs. It is required that the OR signal is generated at a fixed time with respect to the particle passing the hodoscopes, independently of the channels hit. Moreover, for each matrix element the respective signals of the two hodoscope counters have to reach the AND at the same time. 9 output output coincidence H1 information output input H2 H2 & ≥1 matrix element input input input H1 select coincidence (static) information Fig. 7. One matrix element. It can be enabled by setting the select input to a logical 1. The signals H1 and H2 propagate to the corresponding next elements. The information whether at least one enabled coincidence occurred is collected in the coincidence information signal. The scheme presented in Fig. 8 meets both requirements. The 2×32 signals propagate through the matrix with the delays a ,a ,a ,... and b ,b ,b ,... 1 2 3 1 2 3 which are given by the interconnect paths between the LUTs on the FPGA. Signal delays between the mean-timer outputs and the matrix inputs have to be adjusted in order to compensate for the delays inside the matrix as stated in the figure. Consider for example the element marked with (cid:63). Both signals are delayed by a +a +a +b +b before they arrive at the matrix element. Moreover, 1 2 3 1 2 it is easy to see that all signals from input H1 (input H2) reach output H1 (output H2) at the same time. The OR of all 1024 AND outputs carries the logic information whether at least one selected coincidence between H1 and H2 has occurred. The timing information of this occurrence is derived from the output H1. Fig. 9 shows schematically the entire system starting from the LVDS inputs to the outputs. Apart from the output of the matrix needed for the trigger 10