As I described in the last post, the holographic noise can be detected if we measure the correlated noise between two adjacent interferometers. In order to do this effectively, the following requirements were specified:
Digitize two analog signals at 50 MS/s (50 mega-samples per second) from each interferometer. (This is faster than is actually required since we will be looking or the holographic noise near 4 MHz.)
stream all data to disk for offline analysis (2 channels at 50 MS/s = 200 MB/s). (This is critical for traceability and external verification of results.)
perform spectral analysis in real-time for experiment tuning. (Eliminating sources of noise and tuning the experiment will take considerable time and is only feasible if the effect of changes can be observed in near real-time.)
This last requirement is the kicker. Calculating power spectra and cross power spectra for multiple signals in software is time consuming. Today’s computers simply do not have the processing power to perform these calculations at a rate of 50 MS/s. In the past, if a software solution wasn’t possible, the only option was designing custom hardware which wasn’t feasible for most people and most projects. However, there is now a middle ground between software and hardware: Field Programmable Gate Arrays (FPGA). An FPGA is a hardware device that can take on multiple “personalities.” It consists of a variety of logic resources and the ability to interconnect these resources based on the specified description. While the tools to develop FPGAs have come a ways, they are still beyond the grasp of most scientists and engineers.
Let’s take a step back and look at this in a historical context. Fifteen years ago I was designing integrated circuits using VHDL (VHSIC (very-high-speed integrated circuit) description language) in a graduate class. If software wasn’t fast enough, you designed your own hardware. You would simulate and test your design and then send it off to be manufactured. Shortly thereafter, I started working in the field of computer-based measurements when I started at National Instruments developing driver software for DAQ (Data AcQuisition). At that time we just had started supporting the PCI bus along with NuBus (Mac), PCMCIA, and AT (Windows) as well as some other buses I would rather not mention. Our high-end DAQ card was a multifunction-card that could acquire analog signals at 20 kS/s.
Fast forward to the present. A lot of instrumentation is used in the development of this experiment:
However, almost all that is required to satisfy the above requirements in contained in just part of this PXIe chassis:
Slots 2 and 3 contain R-Series devices that are used to run control loops to keep the laser locked. While an incredibly interesting and sophisticated application, it is not related to the above requirements. Slot 6 contains an NI PXIe-5122 digitizer. It is a 2-channel, 14-bit digitizer that can sample at 100 MS/s. While we only need to sample at 50 MS/s, we actually run at 100 MS/s because that allows us to leverage the built-in 35 MHz antialiasing filter. The binary data is streamed from the digitizer, to the controller (slot 1), and then to the NI HDD-8265 12-drive RAID array (not pictured). These devices satisfy the first two requirements quite well. As I mentioned, the most challenging and interesting part of this application is computing the power spectra and cross power spectrum at 50 MS/s. In slot 7 is an NI PXIe-7965R FlexRIO device which contains the largest FPGA available from National Instruments. Often this device is used in conjunction with an analog front-end module. However, since we already had the 5122 and wanted to take advantage of the 5122’s calibration, filtering, and synchronization features; we used the 5122 as the analog front end for the FlexRIO device. The 5122 and the 7965R support peer-to-peer streaming. The controller isn’t involved in this streaming and the data doesn’t even leave the bus segment. I programmed the FPGA on the 7965R using the FPGA Toolkit for LabVIEW. This enabled me to write familiar looking LabVIEW code (within certain constraints) and then leverage the LabVIEW compiler and Xilinx tools to produce the FPGA bitfile. The FPGA calculates and accumulates the power spectra for the two channels and the cross power spectrum between them using an optimized and sophisticated algorithm based on the approach used for the GEO 600 experiment. We hope to publish this IP through National Instruments in some way to share it with the community. The accumulated power spectra and cross power spectrum are streamed from the FlexRIO to the controller for normalization, display, and logging.
This past Friday was my last day at Fermilab working on the Holometer experiment. It was quite satisfying to watch the noise floor of the cross power spectrum of the two photodiodes drop as the application ran. While I didn’t finish everything I wanted and expect there are certainly bugs left to find and fix, I at least left them with a solid application that satisfies the requirements. I look forward to stopping in over the next year and seeing their progress!
Disclaimer. I obviously used to work at National Instruments. I have a lot of friends who work at NI. As a shareholder, I want NI to be successful. This post may seem a bit evangelical, but you have to admit that it is pretty amazing what a high school teacher can do in six weeks with off-the-shelf hardware and software.
This post is one in a series about The Holometer experiment and my work at Fermilab in the Summer of 2011:
- Holometer: Holographic Noise
- Holometer: Interferometer
- Holometer: Spectral Analysis
- Holometer: Transverse Jitter
- Holometer: Correlated Interferometers
- Holometer: Computer-Based Measurements