vetoshield:gas [Assay and Acquisition of Radiopure Materials]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
======Gas Handling Systems======

=====Gas Recirculation System=====

====Overview====

The LBCF (Low Background Counting Facility) is surrounded by an active veto shield designed to detect and timestamp individual muons that enter the facility. The shield is made of interlocking aluminum tubes filled with a gas mixture of 90% argon and 10% carbon dioxide held at slightly above atmospheric pressure. In order for muons to be detected, the gas must maintain the uniform 90/10 composition of argon/carbon dioxide. However, the system is not perfectly sealed and it's inevitable that gas will slowly seep out and that oxygen will diffuse in from the surrounding air.  To compensate for this effect and to allow for monitoring and control the system is configured so the gas  //flows// slowly through the tubes.

====Physical System====

The shield is divided into separate panels of tubes. The gas flows into a given panel through a feed line and exits through return line. A schematic representation of the system is shown below. 

{{:vetoshield:vetoshield_gas_recirculation.jpg?900|}}

The return lines from all the panels converge at a single return manifold and all the feed lines diverge from a single feed manifold. Thus, the entire shield can be thought of as a set of fluid resistances arranged in parallel between the two manifolds ((The fluid resistance through a panel is primarily caused by flow restrictors placed within the feed lines. Thus, the pressure within the tubes is very close to that of the return manifold. This makes the return manifold pressure a good indicator of the amount of gas in the system.)). 

After exiting the return manifold, the fluid is drawn through a positive displacement pump and forced through two Deoxo catalytic converters arranged in series. The Deoxos remove any oxygen that may have seeped into the gas through small leaks in the system. However, in order for oxygen to be removed, a small amount of hydrogen gas must be present.  Upstream of the pump, there is a solenoid valve that opens to allow hydrogen to enter from a pressurized bottle.

After passing through the Deoxos, the gas reenters the feed manifold and is dispersed back to the individual panels. During normal operation, the feed manifold has been held at roughly 1 psi above that of the return manifold. This pressure difference causes a flow rate of several liters per minute per panel ((Based on https://fireplace.soudan.umn.edu:8082/Shield/202 the pressure vs. flow relationship for a typical sized panel is p(q) = (6.58e-3)*q^2 + (1.11e-1)*q where p is the pressure difference between the two manifolds and q the flow rate in L/min.)).

The total gas volume of all the shield tubes is approximately 125 m^3.

====Overview of Control System====

All of the valves indicated in the above schematic are computer controlled. Except for the PBV (Pump Bypass Valve), all valves are solenoid valves that can be either fully closed or fully open. The controller opens and closes the solenoid valves as necessary to maintain the desired operating pressures and gas composition. Except for the recirculation valve, the solenoid valves are closed most of the time during normal operation. The  “purge’’ and “top-up” lines are both fed off the main high pressure supply line. The “exhaust” and “chamber exhaust” lines are open to the atmosphere and are used to relieve pressure within the system as necessary.
 
The controller maintains the desired absolute pressure in the tubes by controlling the “top-up” valve. The concentration of hydrogen entering the first Deoxo is maintained in a similar manner by controlling the H2 valve.

The recirculation pump delivers an essentially constant flowrate regardless of various conditions. The purpose of the PBV is to regulate the pressure at the feed manifold by controlling how much flow from the pump passes through the Deoxos and into the feed manifold. Unlike the solenoid valves, the PBV is a throttleable valve that can be anywhere between completely closed and fully open. When the PBV is closed, 100% of the flow from the pump passes through the Deoxos and to the feed manifold. However, as the PBV opens, a portion of the flow is diverted right back to the pump intake, limiting the flow and pressure supplied to the feed manifold. The controller maintains the feed manifold pressure at a user configurable set-point by continuously adjusting the position of the PBV.  In older documentation and notes the PBV may instead be called the MMV (Main Motor Valve).

The controller is a executable written in LabVIEW named “gas_rack_contollerYYYY-MM-DD.exe” and is found in the folder C:/Gas Rack/ on the gas rack computer. A back up of the executable and LabVIEW source code is included at the end of this manual and will be updated periodically as changes are made. Once the .exe file is opened, the front panel user interface will be displayed. See figure below. All user interaction is accomplished via this graphical user interface. To run the controller, click on the white arrow in toolbar in the upper left corner of the window. Once running, the arrow will change appearance and become blackish. Everything on the front panel can be categorized into either monitors or controls. Monitors give the user real time feedback on important system parameters such as pressures and temperatures. Controls enable the user to make certain changes in system operation.

{{:vetoshield:gasrackguiv2015-1-27.jpg?800|}}

The following section explains the individual items seen on the front panel.

__Master Control__- This is used to toggle between auto mode and override. During normal operation, this control should always be in auto mode. In auto mode, the computer is in full control of the system and no action should be required by the user. Override should only be used when troubleshooting. Override allows the user to choose the state of the solenoid valves and the position of the PBV. Avoid leaving the system unattended with the master control left in override.

__Pump Start__- This button starts the recirculation pump. In both auto mode and override, the pump will only start if the PBV is fully open.

__Pump Stop__- This button stops the pump regardless of any condition.

__Pump__-This virtual LED indicates whether or not the pump is currently on.

__Solenoid Valves__- These are the six toggle switches located to the right of the pump controls. In auto mode, they are monitors that display the states of the respective valves ((There is no feedback from the solenoid valves. So the valve states as indicated here are merely the control signals the computer is sending to the valves. If everything is working properly the actual valve state should mirror the control signal and this detail can be ignored.)). Up indicates open and down indicates closed. In override, these six switches become controls allowing the user to manually open and close the valves.
 
__Filter Display__- When this button is pressed, the front panel monitors will be low pass filtered (smoothed). The equivalent RC time constant can be changed by clicking the "Limits and Setpoints" button and then going to the Advanced tab. The gage pressure used to compute the tube absolute pressure is filtered for at least 10 seconds before being displayed or otherwise used by the program regardless of this button's state.

__Override Switches__- These switches are intended to be used in auto mode when a specific safety prevents the pump from running continuously. See the section below for more information regarding the safeties.

__Pump Bypass Valve Demand and Position__- The first item, the three-position demand monitor, denotes whether the Pump Bypass Valve (PBV) is in the process of opening, closing or whether it’s stationary (off). In manual mode, this item changes to a control and the user is able to slide the bar up and down to choose the action of the valve.
The second item, the long bar indicator, displays the current position of the PBV as determined from a feedback signal generated by the valve controller.

__KR__-This is a custom designed sensor which monitors the purity of the gas.  It was designed by Keith Ruddick based on a proportional tube.  The inside of the tube is coated with a radioactive salt to provide a constant source of ionization.  The current through the charged wire is then read out. This gives a good idea of how the gas will be treated by the veto tubes.  The KR can detect most problems in the gas such as incorrect Ar/CO2 ratio and O2 presence. The black arrow on the virtual meter shows the current reading and the red arrows show the pump-stop limits.

__Pressure Monitors__- These five monitors display the pressures at five crucial points in the system. The pump exhaust indicator displays the pressure as measured in the section of pipe between the pump and the recirculation valve. This should be about the highest pressure in the entire re-circulation loop. The feed manifold indicator display the pressure at the feed manifold. The proportional tube indicators display the gage and absolute pressures in the tubes. The gage pressure is measured with a pressure transducer installed on Panel 13 between tubes 14 and 15. The tube absolute pressure is computed as the sum of the gage pressure and the ambient pressure. The ambient pressure is obtained from pressure transducers in the MINOS cavern via the MINOS DCS status webpage at http://farweb.minos-soudan.org/webdcs/dcs_status.html.

The black arrows and the digital displays show current readings. The red arrows show the thresholds, which if exceeded in auto mode, cause the pump the shut down. The orange arrows on the tube gage pressure monitor show the user-selected gage pressure limits. The green arrow on the tube absolute pressure monitor shows the user-selected absolute pressure setpoint.

__Pump Exhaust Temperature__- This monitor displays the temperature of the pipe wall a small distance downstream of the pump outlet. This should be the highest temperature in the system. Note however that this temperature responds very slowly to changing operating conditions. For example, if the pump is turned on after having been off for a long time, this temperature will take over an hour to rise and level off. The dark red bar indicates the current reading and the bright red bar indicates the upper limit which if exceeded causes the pump to shut off. 

__Pump Cabinet Temperature__-This monitor displays the air temperature inside the pump cabinet. This temperature should roughly follow, but be less than the pump exhaust temperature. The dark red bar indicates the current reading and the bright red bar indicates the upper limit which if exceeded causes the pump to shut off.

__Gas Use Estimator__-This displays the percentage of the time the top-up valve is currently commanded to be open. This number is proportional to the rate at which gas is being consumed by the shield. The number sometime varies considerably over a several second timescale. To get a better estimate of gas use, perform a mental average by observing it for a couple minutes. This number is periodically written to file along with other parameters, so one can calculate a more accurate average over a longer period.

__O2/H2 Sensors__-These sensors indicate the O2 and H2 concentrations at the inlet of the first Deoxo and the outlet of the second Deoxo. The red arrows on the O2 meters display the user selected pump-stop thresholds.

__Limits and Setpoints__-Clicking this button opens a new window allowing the user to configure various emergency stop thresholds and operating set-points. See sections below on safeties and adjusting setpoints.

__Sensor Calibrations__-Clicking this button opens a new window allowing the user to adjust the calibration curves for the various sensors. See the section at the end of this manual for more on calibration.

__Quit with Dignity__-Click this button to stop execution of the program in a dignified manner. Quitting this way turns off the pump and closes all the solenoid valves. To confirm the program has stopped executing, make sure the run arrow in the VI toolbar has turned to solid white.

__Configuration Files Locations__- This section of the front panel is deliberately hidden and is accessed by scrolling far to the right. The two configurations files, one for storing limits and setpoints and the other for storing calibrations, must be correctly selected in these two fields. 

The Auto Email and Auto Log sections of the front panel are explained in detail later.

There are additional hardware pump stop and pump start buttons on the real blue panel to the left of the computer. These buttons offer identical functionality to the virtual buttons on the LabVIEW front panel. 


====Safeties====

The controller software has been written so the recirculation pump stops if a potential problem is detected. The criteria under which the pump stops are based on user configurable limits for the various monitors. These limits are minimum or maximum values that a particular monitor should not reach during normal operation. The following is a list of all the pump stop criteria

  *Pump Exhaust Pressure Upper Limit
  *Pump Exhaust Pressure Lower Limit
  *Pressure Rise Upper Limit
  *Pressure Rise Lower Limit
  *Tube Pressure Lower Limit
  *Pump Exhaust Temperature Upper Limit
  *Pump Cabinet Temperature Upper Limit
  *KR Upper Limit
  *KR Lower Limit
  *O2 IN Upper Limit
  *O2 OUT Upper Limit

The "Pressure Rise" is the pressure difference between the feed manifold and the proportional tubes.

The high and low pressure limits for the pump exhaust and the feed manifold are not considered unless the pump has been running for at least 3 minutes. This feature has been added because both these pressures take a couple minutes to stabilize after the pump has been turned on. All other limits are considered immediately upon start up and can thus immediately trigger the pump to stop.

The limits can be changed while the controller is running by clicking on “Limits and Setpoints” on the front panel. A new window will appear and all pump stop criteria is adjustable from the "Pump Stop Criteria" tab. When finished click either “Cancel” or “Accept Values” to close the window. Do NOT close the window by clicking the red “x” in the upper right corner. Doing so will require a restart of the controller program for everything to again work properly. 

In certain situations, one or more safeties may need to be temporarily bypassed. Such is common if the pump has been left off for an extended period of time and an appreciable amount of O2 has seeped into the system near the KR and/or the O2 sensors. In this case the KR and O2 readings may be outside their limits. If the pump has not been running, these readings are not representative of the average gas quality throughout the shield.  A safety can always be bypassed by setting the corresponding limit arbitrarily large or small. For example, if the KR is malfunctioning and reading very low, the KR Lower Limit can be set to 0 or some negative value that it will never reach. Alternatively, there are three override switches, one for the O2 levels, one for the pressure levels and another for the KR. Turing any one or more of these switches to ON will cause the controller to ignore the respective safeties. For example, the same problem with the KR could be addressed by instead turning the KR override to ON without tampering with the lower limit. The “Pressure Override” simultaneously bypasses all four pressure limits and the “O2 Override” simultaneously bypasses both O2 limits.

Unique to the KR override is an auto reset option. When this switch is turned ON, the KR override will stay on only until the reading falls within the range set by the limits or a timeout period has elapsed. If the pump has been off for some time, the KR reading will likely have fallen below its lower limit and the pump cannot be restarted without turning the override on. Turning the override on with the auto reset also turned on causes the controller to ignore the reading only temporarily while the reading is climbing back toward its normal value. The length of the timeout period can be configured by clicking on the "Limits and Setpoints" button and going to the "Advanced" tab. Its the field labeled "KR Auto Reset Delay" and sets the maximum time the KR override will remain on. The start of the timeout period is reset whenever the override switch is toggled or when the pump is started.

===Absolute Limits===

The limits above are configurable. However, based on the integrity of the equipment, some of the limits should never be set beyond certain “absolute” limits. The system is not designed to ever operate beyond these values. The following absolute limits should be observed
  *Tube Pressure Upper Limit. The panels are pressure tested up to only 3.0 psig. Thus the tube pressure should never exceed 3.0 psig. Also, the discharge on the KR is connected to the return manifold. The discharge pressure on a KR unit should not exceed 1 psig ((some of the more poorly built units may burst at even lower pressures)) so if the KR is left connected to the return manifold, the tube pressure should not exceed 1 psig. The pump does not shut down if the tube pressure exceeds the upper limit (which is why its configurable from the Setpoints tab instead of the Pump Stop tab) but the exhaust valve will open to relieve pressure.
  *O2 IN/OUT Upper Limit. The O2 sensors are only designed to handle O2 concentrations up to 1000 ppm. Exposing them to values above this may damage them.
  *H2 IN/OUT Upper Limit. Although the H2 levels are not monitored as a safety in the same way the O2 levels are, these senors should not be exposed to over 2000 ppm H2 or they could lose calibration. To avoid this happening, the h2 target value should always be set well below 2000 (i.e. under 1000). As an additional precaution the pressure regulator on the h2 bottle should be not be set above ~10 psi. If set higher, h2 will enter the system too fast for the controller to  respond to and the h2 levels will grotesquely exceed the target value.

Although absolute limits on other parameters have not been established, any limit should be chosen only extreme enough to sufficiency bracket the normal operating range of the parameter. 
====Auto Email====

When the “Auto Email” switch on the front panel is turned ON, a notification email will be sent upon emergency pump shutdown. The recipients of the message are those listed in the mail list file. The file path is displayed and can be changed using the field located underneath the auto email switch. The message sent will list the state of all pump-stop criteria at the time of shutdown, highlighting the cause of shutdown. 

The user configurable “Time-Delay” is the minimum amount of time the pump needs to have been running for it to send an email upon shutdown. For example, if the time delay is set to 0.5 minutes and the pump shuts down only 28 seconds after it was started, then a notification email will NOT be sent. However, if the pump shuts down 32 seconds after it was started, a notification email will be sent. The purpose of the time delay is to prevent unnecessary messages from being sent shortly after start-up when the user is still in front of the computer. Thus, anytime the pump is started, the user should wait the time delay before leaving the computer.

==== Auto Log====

When the “Auto Log” switch on the front panel is switched to ON, the controller will automatically record all monitor readings and valve states. A log entry will be written every "Period" minutes. Additionally, a log entry will be written at the moment of emergency shutdown provided the pump had been running for the amount chosen for the email time delay. The auto email does not need to be on for an entry to be written at emergency shutdown. All log data is recorded to a daily text file that is automatically labeled with the date. The "Regular Log Folder" field on the front panel allows the user to change the directory these files are written to. Upon viewing a file, one might accidentally leave the file open for a long time period. In this case, DO NOT SAVE the file upon closing. Doing so will cause all entries that were written while the file was opened to be lost. 

At the end of each entry (row) in the output file there will be a parameter called "Measurement Loop Iteration".This parameter represents the number of data samples that have been taken since the program was last started. If this parameter is the same for all repeat entries, the measurement loop is probably stalling while the automatic log loop is iterating. This parameter can also be used to identify issues with program execution. By comparing the difference in this parameter to the time difference between regular entries, the average sample rate can be determined. The execution delay for the measurement loop is set to 50 ms. During normal operation, the sample rate is around 16 Hz. 

In addition to writing regular log entries, the controller program will log pump starting and stopping activity to a separate file called Pump_Status_Log.txt. The location of this file can be changed in the "Pump Log Folder" field on the front panel. Each log entry consists of a date, time and the event which is either a "start" or a "stop". These entries will be written regardless of whether the auto log switch is turned on. 

====Operation and Basic Troubleshooting====

This LabVIEW program was designed to be intuitive and user friendly. Normal operation of the system should involve minimal interaction by the user. All is fine provided the Master Control is left to Auto, the various override switches are off, the pump is running and all monitors have their limits set to reasonable values. 

To maintain the gas quality in the veto tubes, the pump doesn't need to run 24/7. However the pump should run at least 50% of the time each day. The pump can be manually stopped by simply clicking on the Pump Stop button on the program front panel. After the pump is off, the pump bypass valve (PBV) will begin to open. The pump cannot be restarted until this valve is all the way open as indicated by the position monitor on the front panel. This restriction is hardwired into the AC controls and cannot be changed through the software. After the PBV is fully open, the pump can be easily restarted by clicking Pump Start on the program front panel. If the pump has been left off for several hours, an appreciable but non-hazardous amount of O2 will have seeped into the system through small leaks in and around the pump. This may cause the O2 and KR readings to be outside their limits. In this case the pump can still be restarted by temporarily turning on the KR and/or O2 overrides. After the pump has been running for about a minute, the bad pocket of gas will be pumped away through the Deoxos and the O2 reading will return to normal. At this point, the O2 override can and should be turned off. The same is true for the KR but it often takes more than 10 minutes for its reading to return to normal.  If the pump is left off for several days or longer, a good amount of O2 will have seeped into the system and it will take longer for the O2 and KR readings to become normal after the pump is restarted. 

In any case, one should avoid letting the pump run unattended with any of the overrides switched to on. As an exception, the KR override can be left on provided its auto reset switch is also on. 

The program itself should be left running nearly 24/7 regardless of whether the pump is on. Even when the pump is off, the program opens and closes the top-up valve to maintain correct pressure in the return manifold. Of course its ok to temporarily quit with dignity and close the program to do computer maintenance such as installing OS updates. However, avoid leaving the program off more than a few hours at a time. Any event in which the program is not running for more than a couple hours at a time, the data analysis team should be notified that pressure regulation was temporarily lost, usually by submitting an ELOG entry.

When the pump shuts down it's ok to try restarting it as the cause of shutdown may have been an erroneous sensor hiccup. However, if there is a new leak, the o2 levels will remain high and will not start to come down after a few minutes of running. In this case the pump should be turned off as to deter the spreading of O2 from the leak site to other places in the system.

If turning on the pump CAUSES any of the pressures to rise outside their limits, the pump should be turned off immediately. This most likely indicates a mechanical blockage downstream of the pump, such as a jammed closed recirculation valve. The over-pressurization caused by running the pump under these conditions could cause a catastrophic failure and hence a huge leak.  
====Setpoints and Advanced Configuration ====

===Adjusting Setpoints===

In addition to the pump-stop criteria, other parameters can be changed by clicking on “Limits and Setpoints” and then clicking on either the "Setpoints" or "Advanced" tabs. The following quantities are adjustable from the Setpoints tab

__ Pressure Rise Target__-The controller tries to keep the feed pressure greater than the proportional tube pressure by this amount. Increasing this value increases the recirculation flow rate.

__Pressure Rise Allowed Drift__-This value tells the controller how hard it must try to keep the feed pressure at the target value. It is the expected maximum deviation from the target value. Be aware that setting too small may cause problems such as oscillations and instability. 0.03 psi has worked well.

__ Tube Pressure Target__-The controller tries to keep the absolute tube pressure at this value by opening and closing the top-up valve. 

__H2 IN Target__-The controller tries to keep the H2 IN concentration near this value by opening and closing the H2 line valve. This value chosen is somewhat arbitrary but 200 ppm has shown to work well. One should avoid letting the H2 levels ever exceed 2000 ppm because  this can cause the sensor calibrations to be lost. 

__Tube Pressure Lower and Upper Limits__-  The controller will try to maintain a constant absolute pressure in the tubes provided the gage pressure can be kept between these two limits. The controller will not let the tube gage pressure fall outside these limits for the sake of absolute pressure regulation. To increase the fraction of ambient conditions under which the controller will maintain the absolute pressure target, widen these limits by increasing the upper limit and decreasing the lower limit. However, as mentioned above, allowing the gage pressure to get too high will result in rupturing the KR or in extreme cases, damaging the tubes. There is also an un-quantified risk of letting the tube gage pressure get too low. Letting the gage pressure get too low would result in a vaccum at the pump inlet which could draw in excessive air or rupture the pump diaphragms. 

===Advanced Options===
The following additional items are adjustable from the Advanced tab but should only be adjusted if they are fully understood:

__RC Filter Constant__- All sensor signals are low pass filtered in software before being compared to the pump stop criteria. This purpose of this feature is to prevent sudden glitches in sensor output from causing the pump stop criteria to be exceeded. The filter is a discrete implementation that mimics the behavior of a first order RC filter circuit. An RC value around 1.0 seconds has worked well. Larger values will further soften the output but also cause it to respond more slowly to changing input. Smaller values will cause the output to respond more quickly but will also cause it to be more sensitive to noise or glitches in the input signal. Setting this value to zero will prevent filtering altogether. The controller logic uses the filtered signals regardless of the "Filter Display" button on the main GUI.

__KR Auto Reset Delay__- This is the maximum amount of time the KR override will remain on after the pump is started. This is only applicable if the "Auto Reset" switch on the front panel is turned on. 

The tube absolute pressure and the H2 concentration are controlled using similar algorithms. For tuning the control there are the following options for the top-up valve and also for the h2 valve:

__PID Control Constants__-The setpoint minus the actual value from the sensor comprises an error signal which varies in time. The compensation to this error takes the form of a duty cycle which is the percentage of time the valve is open. The compensation is calculated as the sum of three components, one proportional to the error itself, one proportional to the time integral of the error signal and the third proportional to the time derivative of the error signal. The proportionality constants are k_p, k_i and k_d respectively. The units are in "percentage of time valve is open per unit of error".

__Maximum Switching Frequency__- Determines how often the valve is allowed to change state. Higher values will allow the setpoint to be more closely followed but may also cause more electrical noise and shorten the valve life.

__Reset Integral Compensation to Zero__- The lower limit on the error signal integration is reset to the time at which this button is pressed. This option may be useful for troubleshooting or determining control constants. The code has been written with several features to hopefully prevent integral windup so this option should rarely be needed.

Exclusive to the top-up control is the additional option:

__Top Up Method__- This switch controls which solenoid valve is used for topping up with fresh mixer gas. Either the regular top up or the purge valve can be selected. Using the purge valve may result in problems controlling the feed pressure and result in pressure oscillations. To reduce the chance of pressure oscillations with using the purge valve, increase the allowed drift for the pressure rise target and/or the top-up switching frequency.


====File I/0====

The controller program must be able to read/write from/to the following files. 

__Calibration Configurations Settings File__ - A .ini files that stores calibration constants. The controller reads this file at startup and writes to it when changes are made. This file can have any name.

__Limit and Setpoints Configurations Settings File__ - A .ini files that stores  the items that are user configurable under "Limits and Setpoints". The controller reads this file at startup and writes to it when changes are made. This file can have any name.

__Mail List__- A .txt file that lists the email addresses of those who wish to receive  notification upon emergency pump shutdown. The controller reads from this file immediately after emergency pump shutdown if email is turned on. This file can have any name. 

__Regular Log__- A .txt file that is written to periodically. A new file is written each day and is given a name reflecting the current date. The user can only select the folder to which these files are written.

__Pump Status Log__- A .txt file that is written to whenever the pump starts or stops. The user selects the folder and the file name is hardcoded Pump_Status_Log.txt. If the file is not already present in the selected directory, the program will likely create it automatically.

All of these file paths are user configurable from the front panel. The configuration files path fields are intentionally "hidden" and are accessed by scrolling far to the right on the front panel. If running the controller as an executable, the default file names can only be changed by editing the VI and rebuilding the application. 





====Sensor Calibration====

===Overview===

Except for the valve state and absolute pressure monitors, each front panel monitor corresponds to a physical sensor. All sensors are connected to a single circuit board located behind the blue panel. The signal from each sensor is sent as an analog voltage to the computer via a 64 pin PCI connector. The gas rack controller continuously reads these voltages and translates them into their respective quantities. The translation from raw voltage to front panel reading is called calibration. Thus, to calibrate a sensor means to determine the correct mapping from voltage to reading. Calibration is done entirely within the software.

===Applying a Known Calibration===

The temperature sensors have a fixed calibration given by their manufacturer that is hardcoded into the controller code. All other sensors have a user configurable calibration that can be changed by clicking on “Configure Sensor Calibrations” on the controller front panel while the application is running.  A new window will automatically open. The calibration for all these sensors is assumed quadratic of the form

R(V) = a*V^2 + b*V + c

where R is the actual condition at the sensor and V is the raw voltage. The constants a, b and c are user configurable and are changed by editing the appropriate fields within the window. For some sensors, there isn't a field to edit the constant “a”. In these cases the calibration is assumed linear and a=0 is hardcoded into the application. When finished click either “Cancel” or “Accept Values” to close the window. Do NOT close the window by clicking the red “x” in the upper right corner. Doing so will require a restart of the controller program for everything to again work properly. 

===Determining a Sensor's Calibration===

To calibrate a sensor one needs to simultaneously know the sensor output voltage and the actual condition the sensor is exposed to. For example, to calibrate any one of the pressure transducers, its necessary to know the output voltage of the transducer and, at the same time, the actual pressure the sensor is exposed to.  Determining the raw output voltage can be done by using an altogether separate application called “calibration assistant” located in the gas rack folder. When this application is running  the raw output of all user-configurable sensors will be displayed in real-time. The procedure of varying and measuring the actual condition to which the sensor is exposed depends on the type of sensor. The details of such procedures are usually best left to the user. However a method used to calibrate the h2 sensors is outlined at https://fireplace.soudan.umn.edu:8082/Shield/293.
}
====References and Additional Information====

A backup of the Gas Rack folder containing the necessary controller VIs, configuration files and the executable:

{{:vetoshield:gasrackbackupv2015-7-1.zip|}}

The main VI (gas_rack_controller.vi) is in the folder "Controller and SubVIs" folder along with the SubVIs it directly calls. A copy of the exectuable along with the helper files are inside the folder called "builds". Some of the SubVIs outside the "Controller and SubVIs" folder are also needed. There are some unused VIs.

The LabVIEW code was written in LabVIEW 2012. If using the .exe instead of the VI, LabVIEW itself does not need to be installed but you must have the LabVIEW run-time engine (version 2012f3) and DAQmx 9.4.0 installed.

Also included is the firmware code to program the CPLD located on the control board located inside the blue panel:

{{:vetoshield:gas_rack_cpld.zip|}}

For additional information on the gas re-circulation system contact Jeff Gunderson at gund0328@umn.edu or 612-669-7962.

=====Gas Mixing System=====

====Overview====

The Veto shield for the LBCF requires a mixture of 90% Ar to 10% CO2 for proper operation.  This gas is mixed from bulk argon and carbon-dioxide gas by the gas mixer.  This mixer was originally built for use with the Soudan II detector but has been retrofitted to work with the veto shield alone.  This retrofit included changing the ratio of Ar/CO2 it produced as well as replacing the control system with a new computer interface.  

====Gas Mixer====

The mixer itself mixes the 90/10 Ar/CO2 mixture from bulk quantities of pure argon and CO2. The Ar is supplied in 600 lb dewars while the CO2 is supplied in 50 lb liquid cylinders.  The system can accommodate up to 3 argon dewars and 4 CO2 tanks at a time.  There can be one active Ar dewar and one active CO2 tank at any one time.  The Ar dewers are located below the stairs by the west drift exit.  The Ar dewars must by cycled manually using the physical valves which separate the dewars from the main line.  The CO2 tanks on the other hand are capable of being controlled by the computer through t solenoid valves which separate them from the main valves.  It is good practice to have a full CO2 tank on at least one of the positions at all times.  When a CO2 tank is being changed or one is left disconnected the manual valves must be closed to stop gas from leaking out through the open valves (the solenoid valves are unidirectional.)

The mixer starts with a first stage of regulation for each gas which brings the pressure down from the pressures in the tanks to the range of 100 to 200 psig.  From here the CO2 passes through a differential regulator which precisely matches its pressure to that of the Ar.  Both gasses are then passes through the mixer chamber itself.  The mixer chamber is a large tube which is subdivided into many equal cross-section smaller tubes.  There are a number of tubes devoted to both Ar and CO2.  The mixing ratio is achieved by setting the ratio of small tubes that each of the gasses can pass through.  The unneeded tubes are plugged.  From there the now mixed gas passes through another regulator and through the mix control valve into the mix tank where it is stored until needed.  To add more gas to the tank the mix valve is open and gas flows into the tank.  There is a solenoid valve which controls the flow out of the mix tank to the gas racks as well as a dump valve which can be used to vent the tank.

====Control System====

The control system contains three levels of implementation: hardware, programming, and software.  The hardware system consists of a custom printed control board which has direct control over the solenoid valves as well as receives inputs from the NOVA, KR, and pressure transducers.  The programming layer consists of the CPLD control chip and the programming on the chip.  The software layer consists of a LabView interface which monitors all values off the board and controls the mixer.  For the general user only knowledge of the software interface should be required to use the control system. 

===Hardware Control===

A photograph of the control board is shown below. The large connector to the right of center connects to the PCI slot on the computer. The large black square in the center of the board is the CPLD chip.

{{:vetoshield:mixer_control_board.jpg?700}}

The CPLD configuration file is posted at the end of this manual along with the software written in LabVIEW.

===Software Control===

The software controller is a program written in LabView which monitors the status of the gas mixer and controls the process.

**Front Panel**

The front panel appears when the gas mixer control in LabView is opened. See figure below. This is how the user interacts with the entire gas mixing system. The front panel consists of monitors and controls. Monitors give the user real time feedback on important system parameters such as pressures and gas composition. Controls enable the user to make certain changes in system operation.

{{:vetoshield:mixer_front_panel.jpg?900|}}

__Pressure Monitors__ –
This section displays the readouts for the pressure transducers which monitor the status of the gas mixer.  The pressures are displayed on the gauges as the red lines and the values are displayed on digital displays below each gauge.  The set point limits for all values are shown as the gray lines on the gauges.  All pressures are displayed in units of PSIG.  Unless noted, any value which is outside its set point limits will trigger an error and any value which falls outside of its set points will be further indicated by the digital indicator turning red.  

  * Ar tank pressure – This is the unregulated pressure coming out of the argon dewar.
  * CO2 tank pressure – This is the unregulated pressure coming out of the active CO2 tank.  This value out of bounds will not trigger an error.  It will instead cause the system to start to switch CO2 tanks looking for the first tank with sufficient pressure to satisfy the pressure limits.  
  * Tank pressure– This is the pressure in the mix tank.  The limits in this plot are the values where the mixer will start and stop a mix.  There is a hidden set of limits for this value which serve as the warning error trips.  
  * Ar line pressure – This is the pressure of the argon after it is passed through the first stage of regulation.  
  * CO2 line pressure – This is the pressure of the CO2 after is passes through the first stage of regulation, but before it passes through the differential regulator.  
  * Line pressure – This is the pressure of the gas in the main line connecting the gas mixer to the gas racks.

__Gas Purity__ – This section displays the readout from the NOVA and KR.  Like the pressure readouts the NOVA and KR are both readout on the gauge and on the digital readout with the limits shown in gray and the reading shown in red.  The digital displays also turn red if their respective value goes outside the set point limits.  If either of these readings leaves its limits it will trigger the system to enter error mode.  

__NOVA__ – The NOVA is a commercial CO2 monitor which reads out the percent of CO2 in the gas.  The NOVA is connected to the output of the mix tank after the line valve.  

__KR__ – The KR is a custom designed sensor which monitors the purity of the gas.  It was designed by Keith Ruddick based on a proportional tube.  The inside of the tube is coated with a radioactive salt to provide a constant source of ionization.  The current through the charged wire is then readout as the reading of the tube.  This gives a good idea of how the gas will be treated by the veto tubes.  The KR can detect most problems in the gas such as incorrect Ar/CO2 ratio, O2 presence, or insufficient gas flow.  

__Mix Counter__ – This display counts the number of mixes that have taken place.  It is a cumulative running count which is maintained in a file so that even a loss of power in the computer will not reset this count.  In the roughest sense, this display gives an indication of the amount of gas which has been used.  

__CO2 Control__ – This section monitors and controls which CO2 tank the mixer currently has selected to draw gas from.  Whenever the pressure read in the active CO2 tank falls below outside the limits, the program searches through the tanks sequentially until it finds one which satisfies the limits.  In order to allow for time for the line to re-pressurize the tanks will switch no faster than once every 5 seconds.  If it completes 4 full cycles through the tanks without finding a suitable tank it will trigger error mode.  

__Switch CO2__ – This control allows manual advancing of the active CO2 bottle.  This manual switching is not limited to the one every second as the auto switch function is.

__Bottle indicator__ – The LED indicator below the switch CO2 indicates which of the four possible CO2 bottles is the active bottle.  This display is being read from the board and not sent to the board.  

__Configuration controls__ – This section gives the user access to the configuration files to change the limits on the values, the calibrations of the readouts, and the view plots of the recorded values of the gas mixer.  The sub-vi accessible through this window should never be left open or used during a mix because the main program stops while they are called.  

__Config Calibrations__ – This button gives access to a sub-vi which allows the user to change the calibrations of the readouts read by the computer.  In general these values should not have to be changed, but if a component is replaced then these values may be needed to be changed.  

__Config Limits__ – This button gives access to a sub-vi which allows the user to change the values of the set point limits for all values.  

__Plot History__ – This button gives the user access to a sub-vi which plots the history of all values read out by the gas mixer.  The value is plotted in white and the relevant limits are plotted in red and green.  

__Operation Controls__ – This section contains the controls for operating the software.

__Master On__ – This switch controls weather the computer may send out commands to the control chip or not.  This does not affect any of the readouts.  The only exception to this is the signal for the alarm which will be sent regardless of the state of the Master On switch.  When this switch is red it indicates Master On is disabled and when it is green it indicates it is enabled.  

__Error__ – This switch indicates weather the software is in error mode or not.  It can also be used to initiate or leave error mode.  Error mode will be discussed in more detail later.  When this switch is gray it indicates that there is no error while when it is red it indicates the program has entered error mode.  

__Suppress Alarm__ – This control is used only while the program is in error mode.  It can be activated to suppress both the computer and hardware alarms.  It does not cancel error mode or change any other values.  When this switch is gray it is disabled and while it is green it is suppressing the alarms.  

__E-mail__ – This controls whether or not an e-mail is sent out upon an error. 

__Valve Controls__ – This section contains the controls for the three primary valves which control the operation of the gas mixer: the line, mix, and dump valve.  These switches also serve as indicators of which valves are open and closed.  All switches in this section follow the system that if the switch is green the corresponding valve is open and if the switch is red the corresponding valve is closed.  It is not possible to open both the mix valve and the dump valve.  This is blocked in software and hardware.  

__Line Open__ – This indicates the condition of the line valve, but this is not actually a control unless in error mode.  The line valve is normally controlled by the control chip on the board.  This forces the line valve to close whenever the mix or dump valve is opened.  This built in safety can not be circumvented.  

__Mix__ – This indicates the state of the mix valve and can also be used as a manual control to start a mix.  The mix will always start if Master On is on and the tank pressure is below the lower mix pressure limit.  This switch may also be used to stop a mix which is under way so long as the tank pressure is above the lower mix limit.  Finally, this control cannot be activated if the tank pressure is above the upper mix limit.

__Dump__ – This controls the state of the dump valve.  The dump valve allows the user to release the gas from the mix tank and can only be activated by the user.  If the mix valve is activated then the dump switch cannot be operated.  There is the additional safety in hardware which disables all valves if both the mix and dump valves are opened at the same time.    

**Normal Operation mode**

In normal operation mode the program is in full control of the mixer setup.  It monitors all values and acts accordingly.  If the tank pressure falls below the lower set point it refills the tank until the pressure is above the set point.  If the CO2 tank pressure falls out of range then it searches for a tank with sufficient pressure.  The remaining values are monitored but the control system has no direct way to modify them, so the software enters error mode in the case that any value outside its control leaves its bounded range.  

In this mode the Master On, Mix, Dump, E-mail, and Switch CO2 controls are as the disposal of the user.  The user may also set the limits, calibrations or view the history plots.  In operation mode there should never be need for user input to maintain normal operations as long as the argon and CO2 tanks still have gas.  While the program is in this operation mode, the program will record all available values to the file “c:\Mixer Log.txt” along with the limits for each file every hour on the hour.  The values along with the current mix number will also be stored for every mix in the file “c:\Mix Log.txt”.  The current mix number will also be stored in the file “c:\gas mixer.ini.”

Every time that the program is started it loads the values for the mix counter, limits, and calibrations from the file “c:\gas mixer.ini.”  This file allows changes made to the mix count, limits, and calibrations to be remembered from session to session.  If at any point this file is deleted or destroyed and the program is started the program will start but with 2 file not found errors.  The user should select continue for both of these errors and then regenerate the gas mixer.ini file be opening the limit configuration and calibration configuration windows and clicking ok for each of them.  If the file becomes corrupted then it should be deleted and the above process should be followed to replace it.  To reset the value for the mix counter the user should open the ini file and delete the line which starts “mix count=”.  This will cause the mix counter to reset to zero the next time the program is started.  

**Setting the limits**

To set the limits click on the “Config Limits” button.  This will bring up a separate window where the desired values for the limits may be changed. Once the desired value have been entered press ok to approve the changes or cancel to rescind the changes.  The values on all changed dials will be updated as well as the values in the file “c:\gas mixer.ini” which stores the limits.  To clarify, the fields labeled Mix refer to the limits where the mix will start and stop and the fields labeled tank refer to the limits which will trip an error.  Therefore, the tank limits should always be outside the mix limits or the program will enter error mode before the mix starts or stops.  Do not leave this window open for extended periods of time because the primary program is halted while this window is open.  For this reason, do not open this window while a mix or dump is taking place. 

**Setting the sensor calibrations**

To set the sensor calibrations click on the “Config Calibrations” button.  This will bring up a separate window where the values for the span and zero for all values can be adjusted.  All the sensors used for the gas mixer are linear and do not require any higher order adjustments.  When the desired changes have been made, press ok to confirm the changes or cancel to discard them.  The values on all changed dials will be updated as well as the values in the file “c:\gas mixer.ini” which stores the limits.  Do not leave this window open for extended periods of time because the primary program is halted while this window is open.  For this reason, do not open this window while a mix or dump is taking place. 

**Viewing the history Plots.**

To view the history plots press the button labeled “Plot History.”  This will bring up a window with a graph.  It will default to the first plot in the list.  The other plots can be viewed with the ring selector below the graph which can be used to select all plotable values.  The slider bar can be used to shorten the time scale and zoom in on the present.  Press “STOP” to close this window when done.  Do not leave this window open for extended periods of time because the primary program is halted while this window is open.  For this reason, do not open this window while a mix or dump is taking place.

**Error mode**

When the mixer control program enters error mode it is because a quantity which is not in its direct control has gone out of bounds.  This can happen for any one of the following reasons:

  * Ar tank pressure out of range
  * CO2 line pressure out of range
  * Tank pressure out of range (tank limits not mix limits)
  * Line pressure out of range
  * NOVA reading out of range
  * KR reading out of range
  * All CO2 bottles have been cycled through 4 times without finding a tank with sufficient pressure.
 
Upon any of these events triggering error mode the program immediately resets all valves to their default positions (open for line, closed for mix and dump), shuts off the master on and terminates all processes related to the NI 6010 DAQ card.  It then creates a pop-up displaying the status of all values, their limits, and the time when the error was triggered and saves a copy of this same information to the file “c:\Mixer Error log.txt.”  If the “E-mail” switch is checked the computer also sends a copy of this information to every e-mail address listed in the file “c:/maillist.txt.”  After the e-mail is sent the e-mail switch is deactivated to prevent unintended spam.  The mail list file is not automatically created and must be manually created if it is not present already (simply a text file with each e-mail address listed on a separate line.)  The NI 6010 tasks are then restarted and the controller continues in error mode.    

While in error mode the function of the system is different than in normal mode.  First off the alarm will sound when the program is in error mode.  It can be silenced by toggling the “suppress alarm” button on the operation controls section.  This mutes the alarm, but dose not make any further changes.  Error mode starts with master on turned off and all valves in their default positions.  The controller will take no further actions until the user reactivates master on.  All bottle changes/repairs required should be made prior to reactivating the master on.  

Once master on is reactivated, the controller will resume.  In error mode the only safeties used to regulate the mix are the tank pressure, Ar tank pressure, Ar line pressure, CO2 tank pressure, and CO2 line pressure.  The other quantities are not used for controlling.  This allows the system to mix a new tank even if the KR or another value is out of range.  In error mode, the user still has the ability to dump the tank or manually mix or stop a mix.   The manual mix command is still limited by the same quantities as listed above.  

Error mode can be deactivated when all values have returned to within their designated bounds.  This is accomplished by toggling the switch “Error” on the operation controls section.  If the user tries to exit error mode before all values are within their limits then the system will simply re-enter error mode.  Once error mode is exited the controller is back in full regulated control and no further input is required to resume normal control.

====References and Additional Information ====

A backup of the Gas Mixer folder containing necessary software needed to run the controller:

{{:vetoshield:gas_mixer_2015-7-15.zip|}}


The main VI (mixer_controllerYYYY-MM-DD.vi) is included along with the necessary SubVIs. A copy of the pre-compiled executable along with a few helper files is alongside these files in another folder called “builds”. Also included are the necessary auxiliary files the controller needs access to which are the three log text files it writes to plus the config file "gas mixer.ini" and the mail list it reads from. 

The LabVIEW code was lasted edited in LabVIEW 2012. If using the .exe instead of the VI, LabVIEW itself does not need to be installed but you must have the LabVIEW run-time engine (version 2012f3) and DAQmx 9.5.1 installed.

The CPLD firmware is also included:

{{:vetoshield:gas_mixer.zip|}}

There is an up-to-date version of the .jed configuration file. The source code included is not the most recent version but is the only version I have been able to find.
User Tools

Page Tools
