Review of quantitative accuracy assessments with fast Fourier transform based method (FFTBM)

Borut Mavko; Andrej Prošek

In 1988 United States Nuclear Regulatory Commission approved the revised rule on the acceptance of emergency core cooling system (ECCS) performance. Since that there has been significant interest in the development of codes and methodologies for best-estimate loss-of-coolant accident (LOCAs) analyses. Several new best estimate plus uncertainty methods (BEPUs) were developed in the world. The purpose of the paper is to review the developments in the direction of best estimate approaches with uncertainty quantification and to discuss the problems in practical applications of BEPU methods. In general, the licensee methods are following original methods. The study indicated that uncertainty analysis with random sampling of input parameters and the use of order statistics for desired tolerance limits of output parameters is today commonly accepted and mature approach. (author)

In the past ten years, the Working Group for the Analysis and Management of accidents (WGAMA) initiated activities to promote the use of CFD for Nuclear Reactor Safety (NRS). Best Practice Guidelines were written which are applicable to single phase CFD. Assessment requirements were also addressed. These activities provided more confidence in the application of CFD for safety by defining some conditions and requirements. However, no applicative methods were written about a possible quantitative evaluation of the uncertainty of predictions which is mandatory for complementing a Best-Estimate approach within a nuclear reactor licensing framework. A new activity reported here was then initiated to review the methodologies for determination of the uncertainty of CFD predictions applied to reactor thermalhydraulics. An OECD report was written which is summarized in this paper. A proposal for CFD UQ global approach shows the link between PIRT, Verification, Validation, and Uncertainty Qua...

For the past 15 years, the Technical University of Catalonia (UPC) has been involved in analysis and support to exploitation tasks related to the thermal-hydraulic behavior of commercial nuclear power plants. There has been a close cooperation with 3 Catalan NPPs (Ascó-I and II and Vandellòs-II), all of them 3-loop PWRs of Westinghouse design. More recently, the research activities of

Nuclear Engineering and Design 217 (2002) 179– 206 www.elsevier.com/locate/nucengdes Technical note Review of quantitative accuracy assessments with fast Fourier transform based method (FFTBM) Andrej Prošek a,*, Francesco D’Auria b, Borut Mavko a a ‘Jožef Stefan’ Institute, Jamo6a 39, 1000 Ljubljana, Slo6enia b Uni6ersity of Pisa, Via Diotisal6i 2, 56 100 Pisa, Italy Received 12 June 2001; received in revised form 10 January 2002; accepted 7 April 2002 Abstract In the past 10 years various methodologies were proposed to evaluate the uncertainty of BE code predictions. One common step to all methodologies is the use of experimental and plant data for the nodalization development and qualification. When thermal– hydraulic computer codes are used for simulation the questions raised are: ‘How long improvements should be added to the model, how much simplification can be introduced and how to conduct an objective comparison?’ The proposed fast Fourier transform based method (FFTBM) assists in answering these questions. The method is easy to understand, convenient to use, user independent and it clearly indicates when simulation needs to be improved. The FFTBM shows the measurement– prediction discrepancies— accuracy quantification— in the frequency domain. The acceptability factor for code calculation was determined based on several hundreds of code calculations. The FFTBM method has been applied to various international standard problems, standard problem exercises and other experiment simulations that are presented in the paper. The result shows that the quantitative comparison between thermal– hydraulic code results and experimental measurements with qualitative evaluation may assist the decision whether or not the simulation needs to be improved. © 2002 Elsevier Science B.V. All rights reserved. 1. Introduction The assessment process of large thermal – hydraulic BE computer codes aims principally at verifying their quality by comparing code predictions against experimental data gained mainly by tests performed on scaled plant experimental facilities. The merit of these predictions, adopted for * Corresponding author. Tel.: + 386-1-5885450; fax: + 3861-5612335 E-mail address: andrej.prosek@ijs.si (A. Prošek). nuclear power plant safety analyses, depends on many factors involving code features and user experience. Best estimate (BE) thermal – hydraulic system codes predict reactor transient scenarios as realistically as possible by approximating the physical behavior with some accuracy. Information on inaccuracy of predictions comes from the code assessment and validation process. Extensive experimental programs have been conducted in order to support the development and validation activities of BE thermal – hydraulic codes. A 0029-5493/02/$ - see front matter © 2002 Elsevier Science B.V. All rights reserved. PII: S 0 0 2 9 - 5 4 9 3 ( 0 2 ) 0 0 1 5 2 - 8 180 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 methodology suitable to quantify the code accuracy (i.e. the code capability to correctly predict the scenario observed during the test performed on a scaled facility) has been developed (D’Auria et al., 1994). It is an integral method using the fast Fourier transform (FFT). The fast Fourier transform based method (FFTBM) is easy to understand, convenient to use, user independent and it clearly indicates when simulation needs to be improved. The FFTBM shows the measurement–prediction discrepancies in the frequency domain. The purpose of the paper is to present the applications of the FFTBM to the calculation analyses of International standard problems (ISP) or standard problem exercise (SPE) organized by CSNI or IAEA (i.e. ISP 21 (OECD-NEA, 1989), ISP 22 (Ambrosini et al., 1991), ISP 27 (D’Auria et al., 1994, Prošek and Mavko, 1997), ISP 33 (Purhonen et al., 1994), ISP 35 (D’Auria et al., 1995), ISP 39 (D’Auria and Galassi, 1997), ISP 42 (Aksan et al., 2001) as well as Institut ‘Joz' ef Stefan’ (IJS) calculations of IAEA-SPE-2 (Frogheri et al., 1995) and IAEA-SPE-4 (Prošek et al., 1995), DCMN calculations of IAEA-SPE1– 4 (D’Auria et al., 1996), participants to IAEASPE-4 calculations (Mavko et al., 1997) and other applications (Prošek and Mavko, 1995; Kljenak and Prošek, 1996; Prošek et al., 1996; D’Auria and Ingegneri, 1997; D’Auria et al., 1997b). Second aim was to show the maturity of the method and its usefulness to the thermal– hydraulic code analysts. forth between these two representations by means of the Fourier transform equations, & & F0 ( f ) = F(t)e2yift dt − F(t)= F0 ( f )e − 2yift df (1) − In the most common situations, function F(t) is sampled at evenly spaced intervals in time. Suppose that we have N consecutive sampled values Fk F(tk ), tk k~, k = 0, 1, 2, …, N−1 (2) so that sampling interval is ~. The approximation of the integral in Eq. (1) by discrete sum gives: F0 ( fn )= & N−1 − F(t)e2yifn t dt: % Fk e2yifn tk~ k=0 N−1 =~ % Fk e2yikn/N (3) k=0 where, fn n/N~. The final summation in Eq. (3) is called the discrete Fourier transform of the N points Fk. Let us denote it by F0 n : N−1 F0 n = % Fk e2yikn/N (4) k=0 The relation Eq. (3) between the discrete Fourier transform of a set of numbers and their continuous Fourier transform when they are viewed as samples of continuous function sampled at an interval ~ can be written as F0 ( fn ): ~F0 n (5) 2. FFTBM overview The formula for discrete inverse Fourier transform, which recovers the set of Fk ’s exactly from F0 n ’s is: 2.1. Fourier transform of discretely sampled data Fk = A physical process can be described either in the time domain, by values of F(t) as a function of time t, or else in the frequency domain, where the process is specified by giving its amplitude F0 ( f ) (generally complex number indicating phase also) as a function of frequency f, with − B fB . For many purposes it is useful to think of F(t) and F0 ( f ) as being two different representations of the same function. One goes back and The discrete Fourier transform can be computed with an algorithm called the fast Fourier transform, or FFT, which is algorithm that rapidly computes the discrete Fourier transform. To apply it, functions must be identified by a number of values that is a power with base equal to 2 and sampling theorem must be fulfilled. The fulfillment of the sampling theorem is required to avoid distortion of sampled signals due to aliasing 1 N−1 % F0 n e − 2yikn/N N n=0 (6) A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 occurrence. The sampling theorem says: ‘‘a signal that varies continuously with time is completely determined by its values at an infinite sequence of equally spaced times if the frequency of these sampling times is greater than twice the highest frequency component of the signal’’ (Lapendes, 1978). Thus, if the number of points defining the function in the time domain is N =2m + 1, then according to the sampling theorem the sampling frequency is: N 2m + 1 1 =fs = 2fmax = = Td Td ~ mental and error signal obtained by FFT at frequencies fn, where (n=0, 1, …, 2m) and m is the exponent (m=8, 9, 10, 11). These spectra of amplitudes together with frequencies are used for calculation of average amplitude (AA) and weighted frequency (WF) that characterize code accuracy. For each variable the AA (variable accuracy) is defined as the sum of error function amplitudes normalized to the sum of experimental signal amplitudes: 2m % D0 F( fn ) (7) n=0 AA = where, Td is the transient time duration of the sampled signal and fmax is the highest (maximum) frequency component of the signal. The sampling theorem does not hold beyond fmax. From the relation in Eq. (7) is seen that the number of points selection is strictly connected to sampling frequency. The FFT algorithm determines the number of points, equally spaced, which is a power with base 2 (N range from 29 to 212). Generally, an interpolation is necessary to satisfy this requirement. The FFTBM application implies analysis window (Td) selection, the number of points (N) determination, the cut off frequency ( fcut) value determination, the fixed frequency (minimum maximum frequency of analysis) value selection and the weights selection. A cut off frequency has been introduced to cut off spurious contributions, generally negligible. The sampling frequency (number of points) in the FFTBM interpolation algorithm is defined based on given fixed frequency. 2.2. A6erage amplitude and weighted frequency The FFTBM shows the measurement– prediction discrepancies in the frequency domain. For calculation of these discrepancies the experimental signal (Fexp(t)) and error function are needed. The error function in the time domain DF(t) is defined: DF(t) =Fcal(t) −Fexp(t) 181 (8) where, Fcal(t) is calculated signal. The code accuracy quantification for a individual calculated variable is based on amplitudes of discrete experi- (9) 2m % F0 exp( fn ) n=0 and a WF is defined as the sum of frequencies multiplied (weighted) by error function amplitudes, normalized to the sum of error function amplitudes 2m % D0 F( fn ) fn WF = n=0 m 2 . (10) % D0 F( fn ) n=0 The basic idea of FFTBM is to quantify the discrepancy with single values, therefore, amplitude and frequency are averaged (see Eqs. (9) and (10)). The purpose of the defined values is not to replace the traditional qualitative analysis but to additionally quantify the calculation. The most significant information given by AA, is the relative magnitude of the discrepancy coming from the comparison between the calculation and the corresponding experimental variable time history. When the calculated and the experimental data are equal then the error function is zero (AA is also equal to zero), characterizing perfect agreement. The WF factor characterizes the kind of error, because its value emphasizes where the error has more relevance either at low or high frequencies. Depending on the transient, high frequency errors may be more acceptable than low (in thermal– hydraulic transient, better accuracy is generally represented by low AA values at high WF values). Nevertheless, in the applications of the method the need was felt for a better definitive role of WF in the quantification of accuracy (Ambrosini et al., 1990). A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 182 The meaning of AA and WF can best be illustrated by an example. The experimental signal and calculated signal are given by: Fexp(t)= {t Fcal(t) = if 0B t515 ! 1.4t 4.5+ 0.5t if if t 55 . 5 Bt5 15 (11) Inserting Eq. (11) into Eq. (8) gives for the error function: DF(t) = ! 0.4t 4.5−0.5t if if t 55 . 5Bt 515 (12) The signals were sampled each second thus obtaining 16 discrete points. These discrete signals were input to the FFTBM algorithm. The maximum frequency selected was 0.5 Hz. According to the Eq. (7) 32 points would be sufficient. However, in the FFTBM algorithm the time signal discrete values are interpolated to get 29 – 212 points, depending on the input highest frequency and the number of discrete points. Therefore, the maximum frequency is equal to 17.066 Hz (N= 512 and Td =15 s). For the chosen cut frequency 0.5 Hz the amplitude spectra consists of eight amplitudes (F0 ( fn ) ={7.500, 4.784, 2.392, 1.595, 1.196, 0.957, 0.798, 0.684} and D0 F( fn ) ={0.002, 1.858, 0.416, 0.319, 0.283, 0.172, 0.159, 0.150}). The frequencies are fn = {0.000, 0.067, 0.133, 0.200, 0.267, 0.333, 0.400, 0.467}. The AA calculated following Eq. (9) is: AA= 0.002 + 1.858 + 0.416 + 0.319 +0.283 + 0.172 + 0.159 + 0.15 7.5 + 4.784+ 2.392 + 1.595 + 1.196 + 0.957+ 0.798 + 0.684 = 3.359 = 0.169 19.906 (13) Similarly, following Eq. (10) and multiplying D0 F( fn ) by fn gives the sum 0.51 and the value of WF is then 0.152 as shown by equation: 0.510 WF = =0.152. (14) 3.359 A small value for WF means that the dis- crepancy between the measured and the calculated trends is more important at low frequencies. When WF is large, the discrepancy comes from various kinds of noise and consequently is less important. When looking at the amplitude spectra of the difference signal we can observe that it is exponentially falling function of the frequency (see Fig. 2). Not only spectra of ‘difference signals’ have been elaborated, but also spectra of various measurements including pressure, differential pressure, flow-rate, and temperature. All these spectra show peaks in a frequency range well below 1 Hz. It can be said that typically chosen maximum frequency is 0.5 Hz (this gives sampling interval 1 s). Due to typically falling amplitude spectra most large amplitudes belongs to frequencies up to 0.1 Hz. This holds for transients lasting a few hundreds and thousands of seconds (e.g. small-break loss-of-coolant accidents (SB LOCAs)) with maximum frequency component below 10 Hz. When transients are shorter (e.g 10 s) according to Eq. (7) and due to the fact that maximum number of the discrete points is limited to 4096 the spectra consist of higher frequency components which are reciprocal to time interval duration. Typical values of WF observed in the SB LOCA applications are from 0.01 to 0.2 when transients last hundreds or thousands of seconds. This can be explained by the fact that the number of points in the interpolation algorithm was fixed between 512 and 4096 points at the development phase. For transients lasting 1000 s according to Eq. (7), the maximum possible frequency is 2.048 Hz and for transient lasting 100 s is 20.48 Hz. When transients are shorter even higher frequencies are involved in the spectra, which as a consequence increase the typical value of WF. This additionally shows the expressed need for a better definitive role of WF. Therefore, primarily AA is used for accuracy quantification. 2.3. Accuracy of code calculation The overall picture of the accuracy for the given code calculation is obtained by defining A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 average performance indices, total weighted AA (total accuracy): Nvar AAtot = % (AA)i (wf)i (15) i=1 and total WF Nvar WFtot = % (WF)i (wf)i (16) i=1 with Nvar % (wf)i = 1 (17) i=1 where, Nvar is the number of the variables analyzed, and (AA)i, (WF)i and (wf)i are AA, WF and weighting factors for i-th analyzed variable, respectively. Each (wf)i accounts for experimental accuracy, safety relevance of particular variables and its relevance with respect to pressure (Bovalini et al., 1992). Weighting factors are shown in Table 1. Each weighting factor (wf)i takes into account (D’Auria et al., 1997a): 1. Experimental accuracy (wexp)i : experimental trends of thermal–hydraulic variables are characterized by uncertainty due to intrinsic characteristics of instruments, the measurement method and different evaluation procedures used to compare experimental measures and the code predictions. 2. Safety relevance (wsaf)i : higher importance is attributed to the accuracy of those calculated variables (such as pressure, peak clad temperaTable 1 Weighting factor components for the analyzed quantities (Bovalini et al., 1992) Pressure drops Mass inventories Flowrates Primary pressure Secondary pressure Fluid temperatures Clad temperatures Collapsed levels Core power wexp wsaf wnorm 0.7 0.8 0.5 1.0 1.0 0.8 0.9 0.8 0.8 0.7 0.9 0.8 1.0 0.6 0.8 1.0 0.9 0.8 0.5 0.9 0.5 1.0 1.1 2.4 1.2 0.6 0.5 183 ture, etc.) which are relevant for safety and design. 3. Primary pressure normalization (wnorm)i : This contribution is given by a factor, which normalizes the AA value calculated for the selected variables with respect to the AA value calculated for the primary pressure. This factor has been introduced in order to consider the physics relations existing between different quantities (i.e. fluid temperature and pressure in case of saturated blow-down must be characterized by the same order of error). The measurement of the primary pressure can be considered highly reliable. Weighting factors have been introduced considering that the quantities that are the object of the accuracy evaluation are not independent from each other. The interdependency among the quantities is complex and is fixed (on the code side) by partial differential equations and by the actual system status (on the experimental side). It is impossible to characterize the function of one quantity versus the others. This function also depends upon the selected transient scenario. However, a limited number of transients were selected consisting in the blow-down phenomenon (performed in the PIPER facility available at University of Pisa) and in LOCA experiments (performed in the LOBI facility available at JRC-Ispra). On this basis, the relation between the error in predicting pressure and the error in predicting the other quantities was identified. Clearly this process had at least two elements of arbitrariness: (a) the choice of transients, (b) the combination of the error to get the final weighting factors available in the Table 1. Nevertheless, the consideration of such weighting factors ‘a posteriori’ (i.e. following the application of the method) produces overall FFTBM results apparently more consistent than those achieved without their use. The weighting factor for the i-th variable, is therefor defined as: (w ) (w ) (w ) (wf)i = N exp i saf i norm i var % (wexp)i (wsaf)i (wnorm)i i=1 (18) 184 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 where, wexp is the contribution related to the experimental accuracy, wsaf is the contribution, which expresses the safety relevance, and wnorm the contribution of primary pressure normalization. This introduces a degree of engineering judgment in the method development not in its application that has been fixed by a proper and unique definition of the weighting factors. The weights must remain unchanged during each comparison between code results and experimental data concerning a same class of transient. 2.4. Methodology for quantifying code accuracy Given qualified user and qualified nodalization scheme, code assessment process involves three steps: the first is selection of an experiment from CSNI validation matrices (Aksan et al., 1987) (or a plant transient), then qualitative assessment, and finally the quantitative assessment. The University of Pisa methodology for quantifying code accuracy is related to those suggested by CSNI (Holmström, 1992) and US INEL (Schultz, 1992). The steps are: subdivision of the scenario into ‘phenomenological windows’; for each phenomenological window methodology requires specification of key phenomena that are distinctive for this class of transients (for example break flow), identification of the relevant thermal – hydraulic aspects (RTA) which are peculiar of each transient (these are events or phenomena consequent to the physical process, for example sub-cooled blow-down) and selection of variables characterizing the RTA (for example average break flow); qualitative assessment of obtained results by visually comparing the experimental and calculated variables trends. Qualitative assessment is done by evaluating and ranking the discrepancies between the measured and calculated variable trends. Assessment results can be subjectively described as: Excellent: code predicts the variable qualitatively and quantitatively. Calculation falls within experimental data uncertainty band. Calculation is qualitatively and quantitatively correct. Reasonable: code predicts the variable qualitatively but not quantitatively. Calculation is not within experimental data uncertainty band but shows correct behavior and trends. Minimal: code does not predict the variable, but the reason for this is understood and predictable. The calculation does not lie within experimental data uncertainty band and at times does not even show correct trends and behavior. Unqualified: code does not predict the variable, and the reason is not understood. Calculation does not lie within experimental data uncertainty band and at times does not even show correct trends and behavior. The qualitative assessment gives the first indications about the calculated predictions. The qualitative assessment phase is a necessary prerequisite for a subsequent quantitative phase. It is meaningless to perform this last phase trough the FFT based method if any RTA is not predicted. The quantitative assessment can be managed by applying method based on the FFT. The most suitable factor for the definition of an acceptability criterion is the total AAtot. With reference to the accuracy of a given calculation, we can define the following acceptability criterion: AAtot BK (19) where, K is acceptability factor valid for the whole transient. The (objective) basis for quantifying the acceptability limits is constituted by the state of the art capabilities of current system BE codes in thermal – hydraulics. Several tens of complex transients including small break LOCA, large break LOCA, long lasting transients, have been analyzed by the method based on the FFT using all the codes available to the international community (RELAP, TRAC, ATHLET, CATHARE, SMABRE); several hundreds of time quantities have been dealt with in such a way. In a number of cases detailed and independent code accuracy evaluations were available (this is the case of International standard problems (ISP) 26 and 27) leading to the conclusion that, an excellent calculation could be characterized by K= 0.3. In the same way, a poor calculation (i.e. a calculation A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 predicting all the relevant experiment phenomena with poor accuracy) could be characterized by K =0.5. It can be noted that: AAtot = 0.3 characterize very good code predictions, 0.3BAAtot 50.5 characterize good code predictions, 0.5B AAtot 50.7 characterize poor code predictions, AAtot \ 0.7 characterize very poor code predictions. The difficulty in getting the value AAtot =0.3 (e.g. very good knowledge of boundary conditions and a very detailed nodalization are necessary) and the demonstration that K =0.4 implies an error level acceptable to experienced code users, brought to the decision of assuming this last value as ‘acceptability limit’ for a calculation. The same criterion can be used to evaluate the code capability in the single variable prediction; clearly, in this case the AA factor is the one evaluated for the addressed variable. In particular, acceptability factor K =0.1 has been fixed for the primary pressure, because of its importance. It should be noted that the FFT based method does not allow the identification of the error origin (i.e. user effect, wrong initial condition, nodalization model deficiency, etc.) or to take into account directly time shift of certain phenomena in the quantitative analysis. Time shift is better characterized through the necessary qualitative assessment that must be associated with quantitative assessment. 3. Examples of FFTBM Two FFTBM application examples are presented to show how the method works. The first example shows, how to calculate AA and WF for a single variable prediction (secondary pressure of BETHSY 6.2TC). In the second example, it will be shown how to assess the total code accuracy from the average amplitudes of the selected variables to be considered in BETHSY 6.2TC calculation. 185 Fig. 1. Experimental, calculated and error time trend for secondary pressure. 3.1. Example of calculating AA and WF In this example the FFTBM was applied to the BETHSY 6.2TC calculation of secondary pressure (Hrvatin et al., 2000). The BETHSY 6.2 TC test simulates a scaled 15.24 cm (6 inch) cold leg break transient with no high and low head safety injection and with only accumulator in the intact loop available. During transient after scram and the condenser isolation the pressure increases. The pressure increase is limited by atmospheric relief valves at 7.2 MPa. In the RELAP5 model the relief valve was modeled with time dependent junction which discharges mass without any delay. Later the pressure in the calculation decreases due to cool-down of the primary system while feedwater was isolated. In the experiment the pressure decrease is much faster and it seems that there was additional cooling mechanism not modeled with RELAP5 causing differences in the results. When calculating the AA and WF for secondary pressure shown in Fig. 1 the result is amplitude spectra shown in Fig. 2 for the lowest frequencies. Now the question is how to select maximum and cut frequency. Cut frequency is the frequency, beyond which the amplitudes are not taken into account for calculating AA and WF. We can see that AA and WF are characterized by large amplitudes at low frequencies and that higher frequency components have smaller impact on the AA. The impact of the cut frequency (generally negligible) is shown in Fig. 3. For the selected variable the highest AA is obtained tak- 186 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 Fig. 2. Amplitude spectra for secondary pressure (p47) of BETHSY 6.2TC. ing into account all frequencies. When only half frequencies are taken into account, the difference in results is about 7%. For some variables the highest AA are reached at low cut frequencies and the AA decreases with increasing cut frequency (D’Auria et al., 2000). Therefore, care must be taken into account when selecting cut frequency. On the other hand, the WF monotonically increases with cut frequency. An example is shown in Fig. 3. Therefore, higher cut frequency gives higher WF values and the opposite. Please note that in the analysis the same cut frequency is used for all calculations compared. 3.2. Example of calculating total accuracy Generally, the applications of the FFTBM methodology are performed for 20–25 time trends. Fig. 4. Visual comparison for BETHSY 6.2TC experiment. Fig. 3. Impact of cut frequency on AA and WF. In the present case 23 time trends were selected for accuracy quantification of BETHSY 6.2TC experiment (Hrvatin et al., 2000). Some trends are shown in Fig. 4 together with calculated average amplitudes to show relation between discrepancy and quantitative measure. All selected variables and calculated AA and WF are shown in Table 2. The total AA and WF were calculated using Eqs. (15) – (17) and weighting factors from Table 1. The total AA for four time intervals range from 0.09 to 0.28 all meeting acceptability criterion. A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 187 Table 2 Example of calculating total accuracy for BETHSY 6.2TC Number Variable description Time interval of analysis 0–140 s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Pressurizer pressure SG pressure Accumulator pressure Core inlet fluid Core outlet fluid Upper head fluid Integral break flow SG DC bottom flow Break flow rate ECCS integral flow Heater rod temperature (bottom) Heater rod temperature (middle) Heater rod temperature (bottom) Primary side total mass Core level SG DC level DP inlet-outlet SG Core power DP loop seal broken loop (downflow) DP loop seal broken loop (upflow) DP in pressurizer DP SG inlet DP across DC-UH Total 0–350 s 0–980 s 0–2687 s AA WF AA WF AA WF AA WF 0.08 0.07 0.01 0.01 0.01 0.02 0.03 0.01 0.35 0.00 0.04 0.14 0.02 0.04 0.16 0.63 0.70 0.06 0.27 0.06 0.04 0.07 0.04 0.05 0.09 0.08 0.10 0.04 0.00 0.05 0.10 0.05 0.08 0.07 0.07 0.05 0.05 0.04 0.06 0.08 0.03 0.01 0.01 0.02 0.06 0.01 0.39 2.02 0.04 0.18 0.30 0.11 0.50 0.53 0.81 0.04 0.25 0.05 0.06 0.13 0.04 0.06 0.08 0.07 0.08 0.06 0.19 0.06 0.09 0.09 0.05 0.06 0.07 0.04 0.03 0.03 0.06 0.14 0.06 0.04 0.02 0.28 0.04 0.03 0.45 0.12 0.04 0.14 0.20 0.15 0.58 0.56 0.90 0.04 0.25 0.05 0.06 0.03 0.02 0.04 0.06 0.05 0.09 0.06 0.06 0.06 0.05 0.02 0.04 0.05 0.08 0.05 0.03 0.04 0.06 0.27 0.09 0.11 0.40 0.61 0.04 0.09 0.52 0.14 0.08 0.50 0.24 0.13 0.46 0.54 0.89 0.08 0.27 0.05 0.05 0.04 0.05 0.05 0.05 0.04 0.08 0.05 0.04 0.05 0.05 0.03 0.03 0.05 0.06 0.04 0.06 0.03 0.34 0.11 0.35 0.04 0.35 0.04 0.37 0.04 0.16 1.03 0.41 0.09 0.05 0.04 0.05 0.07 0.17 0.83 0.73 0.20 0.05 0.08 0.06 0.07 0.18 1.20 0.74 0.17 0.05 0.06 0.05 0.05 0.21 0.97 0.88 0.28 0.06 0.06 0.05 0.05 In Fig. 5 is shown influence of selecting cut frequency on total accuracy when fixed frequency 0.5 Hz was selected. When cut frequency is half or double of fixed frequency the AA changes by less than 3%. In this case the total AA increases (the total accuracy decreases) with cut frequency which means that taking into account all frequencies gives the worst result. However, it should be noted that single variables accuracy may be increasing and decreasing with increasing cut frequency, therefore, the variation of total accuracy is smaller than single variable accuracy variations. spread from SB LOCA to other accidents (including severe accidents) and that some level of maturity was reached after one decade of applications. 4. Review of FFTBM applications The review of FFTBM applications shows the historical progress of the method, how the use Fig. 5. Impact of cut frequency on total accuracy for BETHSY 6.2TC experiment. 188 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 ables selected FFTBM shows two different levels of accuracy, which was confirmed by other methods also. Direct comparison with WF results for different calculations has meaning only for similar average amplitudes. In that case, the AA with larger WF is more acceptable result. Therefore, in our case better value of WF for TRAC/PF1 than for R4/M6 for residual mass do not change conclusions based on AA. Ambrosini et al. (1990) felt the need for a better role of WF in the quantification of accuracy. Fig. 6. Results of the FFTBM application to ISP-18. 4.2. Application to ISP 21 4.1. Partial application to ISP 18 Due to its promising characteristics, the FFTBM method was then used to quantify accuracy of blind pre-test calculations of ISP 21 with the title ‘PIPER-ONE Test PO-SB-7 on SB LOCA in BWR- Recirculation line’. The main objectives of ISP 21 were simulation of the PIPER-ONE facility under the conditions of a SB-LOCA in the down-comer region, confirmation of code capabilities describing a test in a geometrically simplified apparatus and the test PO-SB-7 to serve as counterpart test to BWR-related tests in other facilities. In the analysis (Ambrosini et al., 1990), ten variables were considered for seven calculations and the transient duration was 350 s. At that time no procedure was available for weighting factors, therefore, it was decided to put the same weight to each variable. The results for single variable average amplitudes and total accuracy are shown in Table 4. The quantitative analysis was found consistent with conclusions reached in the Compari- After testing on sample curves, the FFTBM was first time applied to pre-test blind calculations (Ambrosini et al., 1990) of ISP 18 with the title ‘LOBI-MOD 2 Small Break LOCA Experiment A2-81’. The main objective of ISP 18 was simulation of a 1% SB LOCA in the non-nuclear LOBI test facility at Ispra with high pressure injection system (HPIS) partially in operation. For the quantitative analysis two variables were selected: intact loop hot leg density and primary system residual mass. The results obtained are presented in Fig. 6 and were compared with other methods also (Ambrosini et al., 1990) whose results are presented in Table 3. It can be noted that, in this case that quantitative information provided is consistent for the various methods, showing a better behavior of the first generation code (RELAP4/MOD6) with respect to more advanced. When comparing AA for the two vari- Table 3 Results obtained by accuracy evaluation methods for selected variables of ISP 18 Variable Intact loop hot leg density Primary side residual mass Method Dm IA MFE Dm IA MFE Code used R4/M6 R5/M1-C R5/M2 TRAC/PF1 0.29 0.65 4.07 0.14 0.2 0.37 0.66 0.89 18 0.12 0.18 1.17 0.65 0.81 16.67 0.27 0.43 4.57 0.66 0.89 28.76 0.22 0.43 4.43 Table 4 Results obtained by FFTBM for double-blind ISP 21 calculations Variable TF03: lower plenum temperature TF17: steam dome temperature CL70: collapsed liquid level in the down-comer MF43: break mass flow rate MF48: LPCS mass flow rate TR50: rod temperature (level B) TR52: rod temperature (level D) TR55: rod temperature (level G) RM65: fluid mass in the loop Average weight Standard weightb a b ABB Ansaldo ENEL JAERI VTT1 VTT2 DCMNa GOBLIN-EM RELAP5/ MOD2 RELAP4/ MOD6 THYDE-B1/ MOD2 SAMBRE (1) SAMBRE (2) RELAP5/ MOD2 AA WF AA 0.084 0.063 0.089 0.109 0.065 0.382 0.196 0.071 0.480 0.149 0.053 0.145 0.266 0.081 0.314 0.266 0.081 0.315 0.038 0.094 0.077 WF AA 0.135 0.539 0.149 0.626 0.130 0.619 0.140 0.444 0.137 0.149 0.137 0.150 0.159 0.392 WF AA 0.155 0.326 0.139 0.108 0.148 1.102 0.150 0.194 0.040 0.253 0.042 0.253 0.156 0.077 WF AA WF AA WF AA 0.086 0.736 0.099 0.385 0.123 0.072 0.131 0.588 0.115 0.351 0.133 0.085 0.140 0.772 0.124 0.274 0.118 0.332 0.084 0.667 0.109 0.432 0.065 0.102 0.063 0.790 0.116 0.876 0.122 0.287 0.063 0.790 0.116 0.880 0.122 0.288 0.171 0.534 0.147 0.106 0.140 0.066 WF AA 0.161 0.152 0.061 0.133 0.094 0.625 0.108 0.167 0.110 1.575 0.110 1.473 0.069 0.117 WF AA 0.133 0.394 0.126 0.455 0.060 0.740 0.087 0.485 0.111 1.106 0.107 1.038 0.113 0.314 WF AA WF AAtot WFtot AAtot WFtot 0.071 0.113 0.128 0.289 0.115 0.260 0.127 0.087 0.282 0.157 0.312 0.116 0.338 0.120 0.062 0.600 0.157 0.574 0.111 0.578 0.111 0.077 0.285 0.150 0.307 0.102 0.279 0.115 0.095 0.646 0.162 0.626 0.104 0.603 0.101 0.087 0.647 0.162 0.610 0.103 0.580 0.100 0.077 0.029 0.102 0.175 0.123 0.180 0.122 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 PA01: lower plenum pressure Participant and code used Post– test calculation. Analysis performed in year 2001. 189 190 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 son report of ISP 21 (D’Auria et al., 1989) on the basis of qualitative observation of the submitted curves. The considerable improvements achievable in accuracy by the tuning of variables affecting code behavior during post-test analysis was further interesting outcome. In this analysis the RELAP4/MOD6 behaves much worse than more advanced RELAP5/MOD2 code. The analysis also revealed the need for development of an objective procedure in order to identify the experimental variables to be considered in the quantification of accuracy and to define the weighting factors for the calculation of AAtot and WFtot. Table 4 shows the results of using standard weights as defined in Table 1. The difference in the results using average weights and standard weights is less than 13%. 4.3. Application to ISP 22 Next FFTBM application was to ISP 22 entitled ‘Loss of Feedwater Transient in Italian PWR (SPES Test SP-FW-02)’. The main objectives were simulation of the integral behavior of the SPES facility under the conditions of complete loss of feedwater to secondary steam separators, core dry out due to lack of heat sinks and opening of pressurizer relief valve and primary side refilling by actuation of emergency feedwater and pressurizer drainage to the primary coolant system. For ISP 22 ‘blind’ pre-test predictions and extensive ‘open’ post-test analyses were performed. The FFTBM analysis was performed in two steps (Ambrosini et al., 1991). The first one was, a qualitative approach to get a preliminary evaluation of the results obtained in each code application. By the visual observation of the curves submitted by participants to pre-test and post-test analyses, the method was aimed to state how much each application can be considered representative of the phenomena actually occurring during the experiment. In the second step, the technique previously applied to ISP 18 and ISP 21 was used (see Sections 4.1 and 4.2). This approach was considered as a pilot study towards the definition of an objective procedure for the quantification of code accuracy. For the accuracy quantification 14 variables were considered. The Fig. 7. Results of the FFTBM application to pre-test ISP 22 calculations as a function of time interval. choice of these variables was made to identify the most representative ones for the characterization of transient evolution, avoiding any possible redundancy. As in application to ISP 21 equal values were adopted for the weighting factors. For pre-test applications, 14 code calculations were analyzed considering three different time intervals, as the submitted curves had different time duration’s (from 0 to 1000 s, from 0 to 4000 s and from 0 to 6000 s). The results for the selected time intervals are shown in Fig. 7. The participants used the following codes: Energoproekt (BULG) used RELAP5/MOD2, CEA (CEAF) used CATHARE 2 code, Institut Jozef Stefan (IJSJ) used RELAP5/MOD2 code, JAERI (JAER) TRAC-PF1 code, Studsvik Energieteknic (STUD) used TRAC-PF1/MOD1 code, University of Pisa (UPIS) used RELAP5/MOD2 code and University of Stratchlyde (USCL) used TRAC code. From Fig. 7, it can be drawn that an analysis of AA as a function of time is necessary to evaluate accuracy of calculations and their acceptability during the various phases of the transient. The reason for greater discrepancies in the second time interval is due to too early emergency feedwater actuation. In the last time interval these discrepancies are averaged over a larger time interval. A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 In the post-test calculations four organizations participated: CEA using CATHARE 2 code (CEA1 and CEA2), University of Pisa (UPIS) with RELAP5/MOD2 code, University of Dresden (UDRE) with RELAP4/MOD6 and BHA BHA Atomic Research Centre from India (INDI) with RELAP4/MOD6. The obtained results are presented in Fig. 8. In addition to the analysis with average weighting factors analysis with standard weighting factors was performed in 2001. General improvement with respect to pre-test results was obtained for all organizations except CEA (partly due to error compensation effect present in the pre-test calculation). The evaluation of post-test results also confirmed that RELAP4 code (and similar ones) were not able to deal with this kind of transient even if the experimental data were available. Reanalysis with currently used standard weighting factors showed that the AAtot are lower (results are better) than in the analysis using average weighting factors. Also it can be noted that for this limited set of variables some calculations fulfil the FFTBM criteria set for total accuracy (AAtot below 0.4). The accuracy of INDI calculation was assessed with only 11 out of 14 variables required. Reanalysis of other post-test calculations with the same eleven variables as for the INDI calculation produces better AAtot values (on the average 0.06smaller values). Fig. 8. Results of the FFTBM application to post-test ISP 22 calculations in the time interval (0 – 6000 s). 191 4.4. Application to ISP 27 Application of FFTBM to the ISP 27 entitled ‘BETHSY Experiment 9.1B; 2ª Cold Leg Break without HPSI and with Delayed Ultimate Procedure’ shows the maturity of the method. The maturity was showed in that the method was sensible in highlighting the differences between pre- and post-test calculations for the same user, normally originated by an ad-hoc code tuning operated in post-test analysis and by the code use at the international level. In this Italian–French application (D’Auria et al., 1994) the full FFTBM method was applied to pre- and post-test calculations as described in the Bovalini et al. (1992). The full FFTBM method include 20–25 variables selected representing relevant thermal– hydraulic aspects and standard weighting factors as shown in Table 1. The main objectives of the ISP 27 were simulation of integral plant behavior under SB LOCA conditions, loop seal clearing and heat transfer during boil-off period, behavior of an uncovered core under reflux-condenser mode of steam generators and primary side refilling by low pressure injection system (LPIS). The blind predictions were requested, however also ‘open’ post-test analyses were processed. In total more than 40 calculations (28 blind, 18 post-test and two open) from 19 countries were submitted and nine different system codes were used. Among the 90 variables selected for the comparison with the calculated results, 25 variables have been selected for a quantitative evaluation of ISP 27 pre-test results. Concerning the post-test submissions, the majority of participants provided only a limited set of variables, considered as significant for the comparison. Taking into account the course of events during the transient and very different time duration of submitted calculations, to compare each other, the results of a considerable number of participants on the basis of common time windows, three analyses have been carried out: from 0 to 3000 s, from 0 to 6000 s and from 0 to 8000 s. The results are shown in Table 5. For each calculation, the participant, country, code and number of variables are specified, and the results for 1/WFtot and AAtot for the three selected time intervals. 192 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 Table 5 Results obtained by FFTBM for pre-test ISP 27 calculations ID Participant/country/code NV Time interval of analysis 0-3000 s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 ZFK Dresden, Germany, R5/M2 KINS, Korea, R5/M3 LTKK, Finland, CATH. 2 V1.2(31.) Kurchatov Institute, Russia, R5/M3 CIAE Beijing, China, R5/M2 JINS, Japan, R5/M2 JAERI, Japan, CATH. 2 V1.2(31.) Studvisk Nuclear, Sweden, R5/M3 NNC, England, NOTRUMP Kurchatov Institute, Russia, R5/M2 JAERI, Japan, CATH. 2 V1.2(21.) Paul Scherrer, Switzerland, R5/M2 GRS, Germany, ATHLET M1.E Tractabel, Belgium, R5/M2 UKAEA, England, R5/M2 VTT, Finland, R5/M3 Texas AMU, USA, R5/M3 UKAEA, England, R5/M3 KAERI, Korea, CATH. 2 V1.2(21.) IPEN/CNEN, Brazil, TRAC/PF1 DCMN/ENEA, Italy–Croatia, R5/M2 AEKI, Hungary, R5/M2 Jožef Stefan Institute, Slovenia, R5/M3 Jožef Stefan Institute, Slovenia, R5/M2 NPPRI, Slovakia, R5/M2 25 25 25 25 25 24 22 25 24 25 22 25 25 25 25 25 25 25 23 22 22 25 25 25 22 0-6000 s 0-8000 s 1/WFtot AAtot 1/WFtot AAtot 1/WFtot AAtot 9.2 10.0 9.4 9.7 8.8 8.6 8.5 8.4 7.3 9.4 8.5 10.0 8.4 8.8 8.6 9.7 9.6 8.6 8.1 8.2 8.0 10.0 8.4 8.4 9.1 0.30 0.68 0.62 0.40 0.31 0.40 0.47 0.25 0.51 0.46 0.47 0.50 0.39 0.41 0.27 0.38 0.39 0.35 0.49 0.35 0.36 0.52 0.38 0.39 0.40 * * * * * * * 10.5 12.3 14.9 11.1 14.3 10.4 14.0 11.5 11.5 12.0 10.6 11.5 12.0 12.6 15.8 14.8 11.9 10.6 * * * * * * * 0.26 0.51 0.60 0.33 0.39 0.36 0.44 0.29 0.33 0.49 0.30 0.41 0.35 0.36 0.55 0.40 0.55 0.56 * * * * * * * * * * * * * * * 10.9 14.2 10.8 11.6 10.8 12.5 14.1 13.8 14.9 9.9 * * * * * * * * * * * * * * * 0.39 0.44 0.30 0.42 0.40 0.38 0.55 0.39 0.44 0.64 *Anticipated prediction of the end of the transient. Note: CATH., R5/M2, R5/M3 correspond to CATHARE, RELAP5/MOD2, RELAP5/MOD3, respectively. In the post-test analysis for only nine calculations all variables were available therefore the analysis with six or seven variables was performed. Table 6 shows that, the results of quantitative analysis with 7(6) variables are different from the analysis with 25(24) variables. The qualitative analysis showed that seven variables are not enough to characterize completely the transient, therefore, the overall accuracy for the addressed calculations. Later another post-test study was performed to compare ISP 27 results obtained by different nodalizations for RELAP5/MOD3.1 code (Prošek and Mavko, 1997). The base and detailed model consisted of 196/207/191/754 and 398/408/396/ 1554 (volumes, junctions, heat structures and mesh points), respectively. The same time windows and variables were used as described above and it was shown that slightly higher accuracy was only obtained for the first windows with the same total accuracy equal to 0.39 in time interval from 0 to 8000 s. Nevertheless, the primary pressure accuracy was fulfilled for the detailed model only. 4.5. Application to ISP 33 First FFTBM application (Purhonen et al., 1994), facilities simulating VVER was to ISP 33 entitled ‘PACTEL Natural Circulation Stepwise Coolant Inventory Reduction Experiment’. The main objectives of ISP 33 were to study natural A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 convection circulation in a VVER plant, series of quasi steady natural circulation periods, single phase natural convection and two-phase natural convection with continuous liquid flow and convection flow under reflux-boiler mode conditions. The ‘double-blind’ pre-test analyses and ‘open’ post-test analyses were performed. Among 21 submitted results, 19 blind calculations have been analyzed with the FFTBM using standard weighting factors (Purhonen et al., 1994). For the analysis five time windows were selected. In selecting variables special attention 193 was paid to their importance with respect to nuclear safety. However, due to unavailability of variables, only nine variables were selected for the analysis. In the post-test calculations all 16 calculations were analyzed. The comparison of blind and post-test calculations was done for ten calculations which are shown in Table 7. It is obvious that, the post-test results have a better accuracy then the pre-test calculations. The obtained results of the accuracy of calculations gives an indication of the capabilities of the codes and code users to predict behavior of this kind of transient. Table 6 Results obtained by FFTBM for post-test ISP 27 calculations ID Participant/country/code NV Time interval of analysis 0–3000 s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 2 4 5 6 8 9 10 18 20 JAERI, Japan, CATH. 2 V1.2(31.) NNC, England, NOTRUMP JAERI, Japan, CATH. 2 V1.2(31.) CIAE (A), China, R5/M2 CIAE (B), China, R5/M2 KINS, Korea, R5/M3 JINS (B), Japan, R5/M2 Kurchatov Institute, Russia, R5/M3 KAERI, Korea, CATH. 2 V1.2(21.) Paul Scherrer, Switzerland, R5/M2 Texas AMU, USA, R5/M3 VTT, Finland, R5/M3 Paul Scherrer (A), Switzerland, R5/M2 Paul Scherrer (B), Switzerland, R5/M2 ZFK DRESDEN, Germany, R5/M2 Jožef Stefan Institute, Slovenia, R5/M3 Jožef Stefan Institute, Slovenia, R5/M2 INEL (open), USA, R5/M3 JINS (A), Japan, R5/M2 Kurchatov Institute, Russia, R5/M2 NNC, England, NOTRUMP CIAE (A), China, R5/M2 CIAE (B), China, R5/M2 KINS, Korea, R5/M3 Kurchatov Institute, Russia, R5/M3 KAERI, Korea, CATH. 2 V1.2(21.) Paul Scherrer, Switzerland, R5/M2 INEL (open), USA, R5/M3 Kurchatov Institute, Russia, R5/M2 6 7 6 7 7 7 7 6 6 7 7 7 7 7 5 7 7 6 7 6 24 25 25 25 25 22 25 25 25 0–6000 s 0–9000 s 1/WFtot AAtot 1/WFtot AAtot 1/WFtot AAtot 9.2 8.0 7.9 8.1 7.9 8.3 7.9 10.9 8.3 8.0 9.1 8.1 7.9 7.8 7.8 9.2 9.7 8.4 7.6 11.4 7.4 9.5 9.1 9.5 9.4 9.5 8.8 8.4 8.8 0.49 0.27 0.31 0.29 0.28 0.39 0.38 0.37 0.46 0.51 0.37 0.42 0.51 0.50 0.33 0.28 0.31 0.41 0.42 0.26 0.42 0.36 0.41 0.45 0.38 0.33 0.27 0.30 0.36 * * * * * * 11.4 14.2 10.7 9.3 9.4 10.2 9.2 8.8 9.3 9.8 10.7 10.9 11.7 11.2 * * * * 14.2 11.5 11.3 12.8 11.6 * * * * * * 0.32 0.37 0.54 0.57 0.45 0.46 0.57 0.57 0.38 0.30 0.32 0.43 0.32 0.33 * * * * 0.43 0.34 0.29 0.33 0.37 * * * * * * * * * * * 9.7 8.9 8.5 9.3 10.0 10.3 10.7 11.5 10.7 * * * * * * * 12.0 11.5 * * * * * * * * * * * 0.52 0.65 0.64 0.56 0.34 0.41 0.52 0.35 0.51 * * * * * * * 0.34 0.39 *Anticipated prediction of the end of the transient. Note: CATH., R5/M2, R5/M3 correspond to CATHARE, RELAP5/MOD2, RELAP5/MOD3, respectively. 194 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 Table 7 Comparison of pre- and post-test calculations for ISP 33 in time interval (0–6600 s) ID 1 2 3 4 5 6 7 8 9 10 a b Participant/country/code Geschellschaft für Anlagen- und Reaktor Sicherheit (GRS) mbH, Germany, ATHLET Mod1 E Hochschule für Tecnik, Wirtschaft und Socialwessen Zittau, Germany, ATHLET Mod1 E University of Pisa, DCMN, Italy, CATHARE 2 V1.3Ea Turkish Atomic Energy Authority, Turkey, RELAP5/MOD3 ver. 5m5 Russian Research Center Kurchatov Institute, Russia, RELAP5/MOD3 ver. 5m5 ‘Jožef Stefan’ Institute, Slovenia, RELAP5/MOD3 ver. 5m5 Nuclear Power Plant Research Institute, Slovakia, RELAP5/MOD2/RMA Nuclear Research Institute, Czech Republic, RELAP5/MOD3 ver. 5m5b Russian Research Center Kurchatov Institute, Russia, SCADAP/RELAP5/MOD2 Research Center Rossendorf Inc., Germany, ATHLET Mod1 E Pre-test Post-test 1/WFtot AAtot 1/WFtot AAtot 92.9 0.21 80.8 0.17 94.4 0.22 80.1 0.19 83.0 121.1 102.1 97.4 104.1 109.0 104.1 81.1 0.29 0.24 0.28 0.19 0.24 0.21 0.30 0.27 103.0 106.1 73.5 101.5 89.9 75.9 85.7 80.1 0.24 0.19 0.38 0.22 0.21 0.15 0.20 0.26 In pre-test CATHARE 2 V1.2E. In pre-test RELAP5/MOD2.5/SRL. 4.6. Application to ISP 35 First application of FFTBM to containment code calculations was to ISP 35 entitled ‘NUPEC Hydrogen Mixing and Distribution Test (Test M-7-1)’. The main objectives of ISP 35 were to study helium (in lieu of hydrogen) distribution phenomena in a model containment, with particular objectives associated to natural convection effects, effects of spray water addition in inner containment regions and validation of containment codes. The ‘blind’ post-test predictions were requested and supplementary ‘open’ post-test calculations were possible. The results calculated by ten pre-test and four post-test participants are summarized here (for details refer to D’Auria et al., 1995). The analysis was performed for one phenomenological window: from 200 to 1790 s. For the analysis 19 variables was selected: dome pressure, temperature in nine different nodes and He concentration in nine different nodes. For the present first application new weighting factors were arbitrary calculated (please note that this is deviation from the original method): one for pressure and as the ratio between node volumes and total volume for temperatures and He concentra- tions. Namely, when the FFTBM started to spread around the world researchers in the severe accident area asked for the possibility to use the method. To use the method the need was definition of weighting factors and acceptability limits for total accuracy. This is responsibility of the containment analysts. Nevertheless, for comparing single variables with experimental trends the FFTBM can be used as it is. The results of application to ISP 35 are shown in Table 8. The definition of weighting factors was just an example of application of the methodology, not accounting for other influences than dome related quantities. With reference to available post-test calculations, the FFTBM analysis, performed in the same window, confirms the better accuracy characterizing the post-test calculations, as a result of improved modeling of boundary conditions, changes in the nodalization and sensitivity analyses. Concerning the global code calculation accuracy, further efforts might be necessary to refine the values of the weighting factor components, to be utilized in the further containment analyses. While comparing the results obtained by CONTAIN code, user effect in code utilization is identified. A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 4.7. Application to ISP 39 Application to ISP 39 is the first application of FFTBM to severe accident. The ISP 39 is entitled ‘FARO-Test L-14 on Fuel coolant interaction and quenching’. The main objectives were benchmarking the predictive capabilities of computer codes used in the evaluation of fuel coolant interaction and quenching phenomena on basis of the FARO test L-14. Particular emphasis was on vessel pressurization, premixing aspects, debris formation and cooling, quenching and steam production rates, quantification of hydrogen formation rate. It was ‘open’ exercise, with experimental results provided. The results for time interval 1.42–4.5 s are shown in Table 9. This application (D’Auria and Galassi, 1997) is peculiar in definition of weighting factors. First the best calculation for pressure (PMED) was identified. This AA value was used as reference to normalize AA for other variables. In this way the ratio between AA for PMED and AA for variable gives the weighting factor for that variable (for example the best ranked is calculation 15, therefore, wf for ENERTOT is 0.096/0.14 =0.69). One of the conclusions of the performed study was that, it confirmed the capabilities of the FFTBM method in ranking generic calculation results. Qualitative evaluation allows and allowed better understanding of the results produced by FFTBM and constitutes a pre-requisite for the 195 application of the FFTBM, as already mentioned. For such analysis the reader is referred to (D’Auria and Galassi, 1997). The use of method is more and more powerful when more and more applications are completed addressing the same phenomena. Addressing the same phenomena and the weighting factors can be assigned on a physical basis. In this sense, the presented application to the study of corium– water interaction should be considered as pilot one. 4.8. Application to ISP 42 The ISP 42 entitled ‘ISP-42- PANDA Test ‘TEPPS’’ main objectives were to study transient and quasi steady– state operation of a passive containment cooling system, primary coolant system and containment system behavior coupled, primary coolant system under low pressure natural convection conditions, steam condensation in the presence of non-condensables, mixing and/or stratification of light or heavy gases in the containment and mixing and/or stratification in large water pools. The ISP 42 was performed in the PANDA facility as a sequence of phases A–F (Auber and Dreir, 1999). The quantitative analysis was performed for the phase A: passive containment cooling system start up. Globally 49 calculations were performed by the participants belonging to ten European organizations. Eight different thermal –hydraulic computer codes were used. Table 8 Comparison of pre- and post-test calculations for ISP 35 in time interval (200–1790 s) ID 1 2 3 4 5 6 7 8 9 10 Participant/Country/Code GRS, Germany, RALOC TRACTEBEL, Belgium, MELCOR 1.8.2 SNL, USA, CONTAIN 1.12 NNC Ltd., UK, COMPACT KEMA, Netherlands, MAPP 4.0 JRC Ispra, CEC, CONTAIN 1.12 JAERI, Japan, CONTAIN 1.12 IVO, Finland, RALOC University of Pisa, Italy, FUMO AEA, UK, CONTAIN 1.12 *Calculations not submitted. Pre-test Post-test 1/WFtot AAtot 1/WFtot AAtot 134.1 134.6 132.3 126.4 143.3 174.8 138.3 121.3 150.2 141.0 0.064 0.056 0.066 0.067 0.055 0.091 0.068 0.047 0.067 0.057 142.4 * * * 128.7 * 175.8 * 127.7 * 0.026 * * * 0.033 * 0.040 * 0.034 * 196 ID 3 15 2 1 11 13 9 10 8 6 12 5 wf 1 0.69 2.2 0.21 2.2 2.1 1.7 Pressure (PMED) Energy (ENERTOT) Steam temperature (TMEDSTEAM) Mixture water level (LTC) Average temperature-elevation 400 mm (TAVG 400×1E-3) Average temperature-elevation 800 mm (TAVG 800×1E-3) Average temperature-elevation 1200 mm (TAVG 1200×1E-3) 0.105 0.096 0.080 0.074 0.065 0.075 0.100 0.061 0.116 0.273 0.065 0.134 0.086 0.140 0.089 0.181 0.286 0.367 0.151 0.418 0.202 0.656 0.524 0.171 0.068 0.043 0.063 0.055 0.109 0.091 0.063 0.047 0.293 0.130 0.281 0.591 0.156 0.463 0.171 0.114 0.102 0.077 0.333 0.147 0.210 0.293 0.211 0.269 0.041 0.044 0.074 0.080 0.052 0.053 0.080 0.084 0.060 0.080 0.096 0.056 0.050 0.047 0.065 0.070 0.040 0.035 0.076 0.062 0.039 0.066 0.073 0.038 0.050 0.056 0.053 0.065 0.044 0.037 0.041 0.088 0.053 0.099 0.091 0.048 AAtot Rank 0.062 0.067 0.070 0.077 0.079 0.079 0.081 0.094 0.123 0.154 0.159 0.187 1 2 3 4 5 6 7 8 9 10 11 12 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 Table 9 Results obtained by FFTBM for ISP 39 in time interval (1.42–4.5 s) A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 197 Table 10 Results obtained by FFTBM for ISP 42. ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Participant/country/code CEA Grenoble, France, ENEA Bologna, Italy, CATHARE Nuclear Research Institute Rez, Czech Republic, RALOC Paul Scherrer Institut (A), Switzerland, RELAP5/MOD3.2 VEIKI Budapest, Hungary, CONTAIN Nuclear Research Institute Rez, Czech Republic, CATHARE Nuclear Research Institute Rez, Czech Republic, CATHARE Paul Scherrer Institut (B), Switzerland, RELAP5/MOD3.2 VEIKI Budapest, Hungary, CONTAIN Paul Scherrer Institut, Switzerland, GOTHIC Paul Scherrer Institut, Switzerland, SPM Paul Scherrer Institut, Switzerland, GOTHIC University of Pisa, Italy, RELAP5/MOD3.2.2 GRS Köln, Germany, COCOYS NRG Arnheim, Netherlands, SPECTRA Taek Ankara, Turkey, RELAP5/MOD3.2.2 Università Politecnica Catalunya, Spain, RELAP5/MOD3.2 Time interval 0–3900 s Time interval 0–5400 s AAtot (10 var.) AAtot (22 var.) AAtot (10 var.) AAtot (22 var.) 0.37 0.19 NA NA 0.19 0.18 0.18 0.19 0.35 0.26 0.26 0.22 0.26 0.33 0.16 0.18 0.17 NA 0.13 NA 0.26 0.12 0.34 0.16 0.22 0.20 0.17 0.17 0.26 0.13 0.18 0.12 0.21 0.20 0.15 0.13 0.14 0.11 0.17 NA 0.17 0.11 0.16 0.17 NA 0.12 0.13 0.11 0.31 0.12 0.30 0.31 0.20 0.08 0.18 0.40 0.27 0.11 0.26 0.27 0.17 0.08 0.18 0.35 NA, calculation ended earlier. For the FFTBM analysis (Aksan et al., 2001) the time intervals selected were from 0 to 3900 s (because three calculations ended around 3900 s) and from 0 to 5400 s. The analysis was performed with ten variables delivered by all participants and 22 variables, significant for the analysis (including liquid masses, pressures, temperatures and mass flows). When not all variables were delivered, it was assumed that the precision of a participant in calculating the not delivered variables would be the average of the precision gained in calculating the delivered parameters. The results for two time intervals and with different number of variables used for total accuracy quantification are shown in Table 10. Again it can be seen that ten variables are not enough to completely characterize the transient. The calculated accuracy is good for most of calculations, both for results obtained by primary system and containment codes. In this study AAtot again proved to be congruent and realistic, providing a valuable, objective and quantitative accuracy evaluation, likely helpful in the code assessment process. 4.9. Application to IJS calculations of IAEA-SPE-2 In the study (Frogheri et al., 1995) 20 variables were selected to characterize the calculation in the three time intervals for post-test calculations using different nodalizations (detailed, middle, simple and miniature with 102/112/99, 71/76/84, 41/43/60 and 30/31/49 volumes/junctions/heat structures, respectively). The results of the FFTBM analysis presented in Table 11 confirm the conclusions previously published in (Mavko et al., 1995) where, it was stated that phenomena during the IAEA-SPE-2 SB LOCA experiment were pre- 198 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 dicted satisfactorily by RELAP5/MOD2 and MOD3 code versions, that even the simplest model produced satisfying results compared with the detailed one and that only down-comer level was predicted wrongly for simple and especially miniature model. Namely, the AA for downcomer level was calculated as 0.35, 0.34, 0.43 and 0.61 for detailed, middle, simple and miniature, respectively. This perfectly agrees with the FFTBM criterion set to 0.4 for single variable. The total AA below 0.3 shown in Table 11 again confirms the conclusion from previous study that the experiments were predicted satisfactorily even with the simplest model. 4.10. Application to IJS calculations of IAEA-SPE-4 This application was performed to IAEA-SPE-4 experiment (Prošek et al., 1995). This is 7.4% SB LOCA, with the break located at the top of the external vessel down-comer, no HPIS available and secondary feed and bleed initiation. The main objectives of the standard problem exercise were to extend the database for VVER-440/213 type NPPS, to validate the computer codes being adapted for VVER safety analyses and to facilitate the exchange of information on application of advanced thermal– hydraulic computer codes. The results of the FFTBM application are shown in Table 12. In this application 22 variables were selected to characterize the calculation in the five time intervals for both pre- and post-test calculations. The post-test calculations were char- acterized with greater accuracy than pre-test. In this calculation, the same input deck was used to compare accuracy of three versions of RELAP5 code. In this case the best results obtained were with RELAP5/MOD2 code. 4.11. Application to DCMN calculations of IAEA-SPE-1 – 4 The study described in (D’Auria et al., 1996) deals with the application to tests performed in PMK facility for the primary pressure accuracy. IAEA-SPE-1 simulates a 7.4% SB LOCA with break located in the upper head of the downcomer, without the hydro– accumulator injection and without high pressure injection system. IAEA-SPE-2 simulates the same LOCA like IAEA-SPE-1 but with available hydro–accumulators. IAEA-SPE-3 simulates a transient originated by the opening of a break located in the upper part of ascending riser in the steam generator. In the study only primary pressure was considered, shown in Table 13. The main conclusions were that, for IAEA-SPE-3 the results were clearly unacceptable, that more complex transient lead to worse results than simple ones (steam generator tube rupture in IAEA-SPE-3), going from pre-test to post-test the assumptions made in the modeling could be adjusted resulting in a better representation of the transient and that advanced codes and recent applications produce better results in the simulation of the considered transients with respect to the precedent ones. Table 11 Application of FFTBM to IJS IAEA-SPE-2 calculations for 20 selected variables Code used Nodalization Time intervals 0–200 s RELAP5/MOD2 RELAP5/MOD2 RELAP5/MOD2 RELAP5/MOD2 RELAP5/MOD3 Detailed Medium Simple Miniature Detailed 0–500 s 0–996 s 1/WFtot AAtot 1/WFtot AAtot 1/WFtot AAtot 21.7 22.32 24.32 20.46 22.97 0.079 0.076 0.066 0.073 0.077 25.38 26.35 26.54 29.19 24.24 0.151 0.141 0.148 0.162 0.137 24.68 24.62 25.5 30.58 26.13 0.169 0.156 0.176 0.204 0.154 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 199 Table 12 Application of FFTBM to IJS IAEA-SPE-4 calculations for 22 selected variables Calculation Time intervals 0–50 s 0–350 s 0–1200 s 0–1400 s 0–1800 s 1/WFtot AAtot 1/WFtot AAtot 1/WFtot AAtot 1/WFtot AAtot 1/WFtot AAtot Pre-test M2A M2B M3.1A M3.1B M3A M3B 14.5 14.5 14.5 14.7 16.1 14.4 0.060 0.070 0.066 0.063 0.072 0.072 23.1 22.1 18.3 21.3 19.2 18.8 0.131 0.138 0.168 0.161 0.164 0.132 23.7 23.8 24.0 24.0 23.5 21.6 0.149 0.160 0.187 0.236 0.188 0.177 23.7 24.0 23.7 23.8 23.6 22.2 0.197 0.207 0.237 0.267 0.238 0.231 27.5 27.6 27.8 27.2 28.0 26.3 0.185 0.194 0.225 0.248 0.222 0.211 Post-test M2 M3.1 M3 14.0 14.3 17.0 0.064 0.070 0.071 22.3 20.7 21.3 0.114 0.152 0.152 22.1 22.2 22.0 0.134 0.172 0.178 22.4 22.9 22.9 0.183 0.226 0.231 26.1 27.4 29.1 0.175 0.214 0.217 M2-RELAP5/MOD2/36.05, RELAP5/M3-MOD3 5m5, M3.1-MOD3.1; A, break on top of the DC (normal location); B, break in the middle of DC. 4.12. Application to IAEA-SPE-4 (Italian– Slo6ene – Hungary– IAEA cooperation) The analysis of code accuracy for IAEA-SPE-4 pre- and post-test calculations is presented in (Mavko et al., 1997) while, qualitative analysis is presented in (Prošek and Mavko, 1996). In total 18 pre-test and 22 post-test calculations were performed by 19 organizations from 15 countries. In Table 14 the results for primary pressure accuracy and total accuracy were compared with for 13 organizations performing both pre-test and posttest calculations. As only 12 variables were available for each participant the analysis is somewhat less complete. However, the limit (0.1) for the primary pressure accuracy allows such a comparison. For the best calculations the same result is obtained by ranking, by pressure accuracy or total accuracy. An exception are CATHARE calculations, one with high pressure accuracy giving relatively low total accuracy (calculation three) and another with poor pressure accuracy (calculation seven) giving quite good total accuracy. Nevertheless, in the post-test, the calculation seven achieved second best primary pressure and the best total accuracy. On the other hand, calculation three achieved the best pressure and was ranked again very low. Most calculations predict primary pressure very good with one exception (calculation 13). According to acceptance criteria, all calculations from Table 14 are very good calculations which shows that the transient was not complex. As similar WFtot were calculated in all cases, the accuracy is judged only based on AAtot results. In general the post-test calculations shows better accuracy than pre-test. All but one participant improve the pressure. However, it is interesting that four calculations not improving the post-test total accuracy were ranked among the six best pre-test calculations. It seems that changes (tuning the results) in the model did not always contribute to better results, but we are limited to make any further conclusions because of limited number of variables used in the analysis. The validity of predictions is also limited by discrepancies highlighted in the qualitative accuracy evaluations. Several calculations did not predict the core dry-out occurrence and should have been excluded from further quantitative accuracy evaluations subject to assessment criteria. However, the dry-out in the experiment constitutes a typical bifurcation phenomenon controlled by minor variations of boundary conditions whose prediction is beyond the capabilities of the present 200 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 generation codes, and the overall transient scenario is not affected by the phenomenon. Both these observations led to the decision to include all calculations in the quantitative accuracy evaluation. 4.13. Application to SB LOCA database The application of FFTBM to SB LOCA database was performed to evaluate accuracy of RELAP5/MOD2 and CATHARE2 code. The studies are described in D’Auria et al., (1997b) and D’Auria and Ingegneri, (1997). The database consists of 12 SB LOCA tests performed in four integral test facilities (LOBI, SPES, BETHSY and LSTF). Additionally, six tests are counterpart tests (BL-34, BL-44, SB-03, SB-04, 6.2 TC and SB-CL-21). The RELAP5/MOD2 was used to simulate all transients while CATHARE was used for simulation of counterpart tests only. The results of FFTBM application are shown in Table 15. In all tests the same set of 24 variables was used. However, as some variables were missing or not available in the database this number vary from 13 to 24. When comparing the results to acceptance criterion all calculations fulfill criterion 0.4 for total accuracy. When comparing RELAP5/MOD2 and CATHARE2 code, the accuracy is similar. Some calculations are better with RELAP5/MOD2 and some with CATHARE2. The mean total accuracy for counterpart test is 0.3 for both RELAP5/MOD2 and CATHARE2. The highest accuracy achieved was 0.14 and the lowest 0.39. 4.14. Application to SB LOCA spectrum In the study by Kljenak and Prošek, (1996) eight different scenarios were selected to compare the AA for 19 variables. Because these variables were not standard, special weighting factors were defined based on the phenomena identification and ranking table (PIRT) process. Additionally, an analysis using equal weighting factors was performed. Eight different break sizes were compared with 5.08 cm (2 inch) break (which was used instead of an experiment): 3.81, 4.13, 4.45, 4.76, 5.40, 5.72, 6.03 and 6.35 cm. This is a special application of FFTBM. The results are shown in Fig. 9, compared with case without weighting (AAtot is mean of single variable AA). Different slope of the line on the left and right from 5.08 cm is partly due to the fact that FFTBM gives different results if we compare with smaller signal to larger or larger to small. Nevertheless, the trends are monotonic which shows that the more similar are the breaks the more similar are also transients. This fact supports FFTBM method in that we can physically expect that the larger are differences in the break size the more different are the transients. 5. Results and discussion The presented results of FFTBM applications suggest that the FFTBM is a suitable method for accuracy quantification of thermal–hydraulic code calculations if the latter can be compared Table 13 Application of FFTBM to DCMN IAEA-SPE-1–4 for primary pressure Experiment Time interval Calculation Code used 1/WF AA IAEA-SPE-1 0–997 s IAEA-SPE-2 IAEA-SPE-3 IAEA-SPE-4 0–847 s 0–1819 s 0–1800 s Pre-test Pre-test Pre-test Pre-test Pre-test Pre-test Post-test Post-test Post-test RELAP4/MOD6 RELAP5/MOD1-EUR RELAP5/MOD2 RELAP5/MOD2 RELAP5/MOD2.5 CATHARE RELAP5/MOD2.5 RELAP5/MOD3.1 CATHARE 7.3 9.8 20.9 11.3 26.8 9.4 24.7 29.2 14.6 0.117 0.157 0.227 0.310 0.091 0.181 0.105 0.067 0.068 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 201 Table 14 Application of FFTBM to IAEA-SPE-4 in time interval (0–1790 s) ID 1 2 3 4 5 6 7 8 9 10 11 12 13 Participant/country/code University of Zagreb, Croatia, RELAP5/MOD2 Lappeenranta University of Technology, Finland, RELAP5/MOD3.1 Institute de Protection et de Sûreté Nucléaire, France, CATHARE2 V1.3E GRS, Germany, ATHLET Siemens AG, Germany, RELAP5/MOD2.5 Electric Power Research Institute, Hungary, MELCOR University of Pisa, Italy, CATHARE2 V1.3E Institute for Atomic Energy, Poland, RELAP5/MOD2 Kurchatov Institute, Russia, RELAP5/MOD3 ‘Jožef Stefan’ Institute, Slovenia, RELAP5/MOD2 ‘Jožef Stefan’ Institute, Slovenia, RELAP5/MOD3 ‘Jožef Stefan’ Institute, Slovenia, RELAP5/MOD3.1 Texas A&M University, USA, RELAP5/MOD3.1 Pre-test Post-test Pressure Total Pressure Total AA WF AAtot WFtot AA WF AAtot WFtot 0.11 0.08 0.03 0.04 0.26 0.22 0.04 0.03 0.09 0.08 0.06 0.05 0.28* 0.23 0.04* 0.03 0.08 0.04 0.31* 0.03* 0.06 0.05 0.29 0.03 0.16 0.13 0.20 0.15 0.07 0.23 0.09 0.10 0.11 0.16 0.03 0.03 0.03 0.05 0.05 0.06 0.04 0.05 0.04 0.05 0.39 0.31 0.35* 0.23 0.20 0.30 0.22 0.24 0.27 0.30 0.03 0.03 0.04* 0.03 0.03 0.04 0.03 0.04 0.03 0.04 0.11 0.09 0.08 0.08 0.08 0.11 0.08 0.09 0.08 0.15 0.05 0.03 0.04 0.05 0.05 0.05 0.04 0.04 0.04 0.06 0.30 0.26 0.24 0.19* 0.22 0.30 0.20 0.25 0.25 0.26 0.04 0.04 0.04 0.03* 0.03 0.04 0.03 0.03 0.03 0.04 *Analysis performed for 11 variables. with experimental results. It was mainly used for SB LOCA calculations. Table 16 shows the lowest calculated total average accuracy for each ISP (among the submitted ISP calculations to which FFTBM was applied) or organization calculation (among several calculations with different code versions). For each test, the time interval (end of transient), the type of calculation (calculation submitted to ISP or SPE, or non-submitted calculation performed by organization later), the code used, the number of selected variables, the primary pressure accuracy and the total accuracy (with standard and average weights) are shown. It can be seen that for primary system calculations the set criterion K=0.4 for total accuracy was achieved while for severe accidents the method is presently limited to ranking of calculations (there are no set criteria). The total accuracy using standard weighting factors ranges from 0.14 to 0.39 for best calculations and may support the acceptability limit K= 0.4 as reasonable criterion. However when observing obtained total accuracy it can be noticed that complexity of the transient is an important factor. The fact that many best ISP calculations were obtained by RELAP5 code can be partly attributed to the fact that it is a widely used code. Table 16 also shows that the tests simulated VVER transients are ranked in the first half. The reason may be less complex transients. This study has taken step also in the direction of defining the relation between results obtained by standard weighting factors and average weighting factors. The correlation factor r=0.92 indicates that when a sufficient number of variables is used in the FFTBM analysis the influence of engineering judgement (in the specification of weighting factors) on the results is less important. When looking at the primary pressure AA the correlation between it and total AA calculated with standard weighting factors is r=0.3 indicating hardly any relation. That means, care must be taken while comparing different transients with primary pressure accuracy only. The proposed method has still some weak points which could be resolved, therefore, further investigations may be performed, especially to give meaning also to the phase spectrum. A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 202 Table 15 Results of FFTBM application to SB LOCA database Facility LOBI LOBI LOBI LOBI LOBI LOBI SPES SPES BETHSY BETHSY LSTF LSTF a b Test BL-06 BL-12 A1-82 A2-81 (ISP-18) BL-34b BL-44b SB-03b SB-04b 6.2 TCb 9.1b (ISP 27) SB-CL-21b SB-CL-18 (ISP 26) RELAP5/MOD2 CATHARE2 V1.3U NV AAtot WFtot 23 23 21 20 24 24 22 22 23 13 21 18 0.29 0.24 0.14 0.28 0.33 0.33 0.32 0.26 0.28 0.39 0.31 0.15 0.02 0.04 0.04 0.04 0.05 0.05 0.05 0.06 0.06 0.08 0.05 0.08 NV AAtot WFtot a a a a a a a a a a a a 24 24 22 22 24 0.29 0.39 0.26 0.28 0.23 a 20 a 0.35 a a 0.05 0.05 0.06 0.05 0.05 a 0.05 a Not available. Counterpart tests. Nevertheless, the obtained results showed that the method is good enough for quick assessments of which variables or phases of the transients need to be improved. One important result is also that even if better methods for code accuracy assessment would be developed the result would still depend on the complexity of the transient. Without this information, setting criteria with respect to what is good or not would be less meaningful. However, such information again requires some engineering judgement. At present moment it seems that the large number of applications is one of the main advantage of choosing existing FFTBM for accuracy assessment of code calculations for experiments. For single variable comparison also other powerful indices of accuracy may be used as proposed by Kunz et al., 2002. This may be more important for code developers. However, even in this case it should not be forgotten that, the improving one parameter might deteriorate the other parameters. Therefore, the information on total accuracy is in favor to choose FFTBM. The applications presented could be a reference database for new users of FFTBM when to stop improvements to their simulation. This is one of the first attempts to present best (as judged by FFTBM) results of several calculations in one place. 6. Conclusions A methodology suitable to quantify the BE codes accuracy with respect to experimental results has been developed. This integral method is using the FFT to represent the code predictionsexperimental data discrepancies in the frequency domain. The weighting factors and acceptability criteria that are part of the evaluation process were set based on best engineering judgement. The AA and WF were calculated for several code calculations of experimental results to estimate Fig. 9. Application of FFTBM to SB LOCA spectrum. A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 203 Table 16 Summary of most accurate ISP, SPE or organization calculations as judged by FFTBM Test Time interval Type of calculation Code used NV AApr AAtot (stand. wf) AAtot (avg. wf) ISP 35b ISP 42 b ISP 39b 200–1790 s 0–5400 s 1.2–4.5 s MAPP 4.0 SPECTRA COMETA 2d-JRC RELAP5/ 19 22 19 NA 0.03 0.11 0.03a 0.08 0.11a NA 0.14 NA 21 0.09 0.14 0.23 MOD2 RELAP5/ 9 NA 0.15 NA 18 0.02 0.15 0.21 20 0.12 0.15 0.22 22 0.08 0.18 0.24 10 0.04 0.18 0.18 11 0.08 0.19 0.30 24 23 0.10 0.06 0.23 0.24 0.33 0.30 22 22 0.08 0.08 0.26 0.26 0.38 0.39 20 0.10 0.28 0.34 24 23 0.12 0.13 0.29 0.29 0.40 0.42 14 0.21 0.30 0.43 25 NA 0.3 NA 21 0.04 0.31 0.44 24 0.04 0.33 0.37 25 0.09 0.39 0.57 13 0.11 0.39 0.41 LOBI A1-82 0–4944 s Best ISP (post-test) Best ISP Best ISP (open exercise) PISA calculation ISP 33c 0–6600 s Best ISP (post-test) ISP 26 0–1000 s PISA calculation IAEA-SPE-2 c 0–996 s IJS calculation 0–1800 s IJS calculation 0–350 s Best ISP (post-test) 0–1790 s Best SPE (post-test) BETHSY 6.2 TC LOBI BL-12 0–2179 s 0–3768 s PISA calculation PISA calculation SPES SB-03 SPES SB-04 0–2034 s 0–1637 s PISA calculation PISA calculation ISP 18 0–4939 s PISA calculation LOBI BL-34 LOBI BL-06 0–2400 s 0–7000 s PISA calculation PISA calculation ISP 22 0–6000 s Best ISP (post-test) ISP 27 0–8000 s Best ISP (pre-test) LSTF SB-CL-21 0–2113 s PISA calculation LOBI BL-44 0–2350 s PISA calculation ISP 27 0–8000 s IJS calculation ISP 27 0–8000 s PISA calculation IAEA-SPE-4 c ISP 21 IAEA-SPE-4 c MOD3 RELAP5/ MOD2 RELAP5/ MOD3 RELAP5/ MOD2 RELAP5/ MOD2 CATHARE2 V1.3E CATHARE2 RELAP5/ MOD2 CATHARE2 RELAP5/ MOD2 RELAP5/ MOD2 CATHARE2 RELAP5/ MOD2 RELAP5/ MOD2 RELAP5/ MOD3 RELAP5/ MOD2 RELAP5/ MOD2 RELAP5/ MOD3.1 RELAP5/ MOD2 Best ISP, it means that this is the pre- or post-test submitted calculation with the lowest AAtot calculated. a Special weights selected. b Severe accident. c VVER transient. quantitatively the accuracy of the respective code predictions. It was shown that most accurate calculations as judged by FFTBM achieved the total average accuracy below 0.4 which was set as ‘acceptability limit’ for calculation (see Section 2.4). 204 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 The application of the method to the international standard problems, standard problem exercises and other experiment simulations showed the FFTBM as a convenient mathematical tool for quantitative assessment of experimental results. The results of such quantitative assessments together with qualitative evaluation may assist the decision whether or not the simulation needs to be improved. Appendix A. Nomenclature ISP IVO JAERI JINS JRC K KAERI KINS LOBI LSTF LTKK D0 F F0 AA ABB ACC AEA ATHLET BETHSY BL CEA CIAE DC DCMN Dm DP ECCS ENEA ENEL f F GRS IA IAEA IJS INEL error function amplitude spectra time function amplitude spectra average amplitude Asea Brown Boveri accumulator Atomic Energy Authority Analyses of Thermo–hydraulics in Leaks and Transients Boucle d’etudes thermo–hydrauliques systeme broken loop Commissariat a l’Energie Atomique China Institute of Atomic Energy down-comer Dipartimento di Costruzioni Meccanice e Nucleari maximum value of error function differential pressure emergency core cooling system Ente per le Nuove tecnologie, l’Energia e l’Ambiente Ente Nazionale per l’Energia Elettrica frequency time function Geschellschaft für Anlagen- und Reaktor Sicherheit (GRS) mbH Index of agreement International Atomic Energy Agency Institut ‘Joz' ef Stefan’ Idaho National Engineering Laboratory MAAP MFE NPPRI NV, Nvar PRZ PWR RELAP SG SNL SPE SPES t T ~ UH UK UKAEA VEIKI VTT VVER WF wf ZFK International Standard Problem Imatran Voima Oy Japan Atomic Energy Research Institute Japan Institute of Nuclear Safety Joint Research Center acceptability factor Korea Atomic Energy Research Institute Korea Institute of Nuclear Safety Loop for Off-normal Behaviour Investigation Large Scale Test Facility Lappeerannan teknillinen korkeakoulu (Lappeenranta University of Technology) Modular Accident Analysis Program Mean Fractional Error Nuclear Power Plant Research Institute number of analyzed variables pressurizer pressurized water reactor Reactor Excursion and Leak Analysis Program steam generator Sandia National Laboratories standard problem exercise Simulatore per esperienze di sicurezza time transient time sampling interval upper head United Kingdom United Kingdom Atomic Energy Authority Villamosenergiaipari Kutató Intézet (Institut for Electric Power Research) Valtion Teknillinen Tutkimuskeskus Voda-Vodianye Energeticheski Reaktoryi weighted frequency weighting factor Research Center Rossendorf Inc. A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 ZF error function Subscripts calc d exp i max norm pr s saf tot calculated duration experimental i-th variable maximum normalized primary pressure sampling safety total Superscripts m power of 2 References Aksan, S.N., Bessette, D., Brittain, I., D’Auria, F., Gruber, P., Holmström, H.L.O., Landry, R., Naff, S., Pochard, R., Preusche, G., Reocreux, M., Sandervag, O., Städtke, H., Wolfert, K., Zuber, 1987. CSNI Code Validation Matrix of Thermo-Hydraulic Codes for LWR LOCA and Transients. CSNI Report No. 132. Aksan, S.N., D’Auria, F., Bonato, S., 2001. Application of Fast Fourier Transform Based Method (FFTBM) to the results of ISP-42 PANDA test calculations: Phase A. Proceedings of the ICONE-9. Ambrosini, W., Bovalini, R., D’Auria, F., 1990. Evaluation of accuracy of thermal – hydraulic code calculations. Energia Nucleare 7, 5 – 16. Ambrosini, W., Breghi, M.P., D’Auria, F., 1991. Evaluation of code accuracy in the prediction of the OECD/CSNI ISP 22, University of Pisa DCMN-NT 176 (91). Auber, C., Dreir, J., 1999. ISP 42 Panda test Phases — History of events, Boundary and Initial Conditions, Calculation parameters, PSI Internal Report TM-42-98-40, ALPHA 835-1, Villingen (CH). Bovalini, R., D’Auria, F., Leonardi, M., 1992. Qualification of the Fast Fourier Transform based methodology for the quantification of thermalhydraulic code accuracy, DCMN Report, University of Pisa, NT 194(92). D’Auria, F., Galassi, G.M., 1997. Accuracy Quantification by the FFT method in FARO L-14 (ISP 39) open calculations, University of Pisa, NT 309(97). D’Auria, F., Ingegneri, M., 1997. Qualitative and quantitative analysis of CATHARE2 code results of counterpart test calculations in LOBI, SPES, BETHSY, LSTF facilities, Proc. of NURETH-8, vol. 2, pp. 1195 – 1203. D’Auria, F., Mazzini, M., Oriolo, F., Paci, S., 1989. Com- 205 parison report of the OECD/CSNI International Standard Problem 21 (Piper-one experiment PO-SB-7), CSNI Report No. 162. D’Auria, F., Leonardi, M., Pochard, R., 1994. Methodology for the evaluation of thermalhydraulic codes accuracy. Proc. Int. Conf. on New trends in Nuclear System Thermohydraulics, Pisa, pp. 467 – 477. D’Auria, F., Oriolo, F., Leonardi, M. and Paci, S., 1995. Code Accuracy Evaluation of ISP35 Calculations Based on NUPEC M-7-1 Test. Proc. Second Regional Meeting: Nuclear Energy in Central Europe, Nuclear Society of Slovenia, Ljubljana, p. 516 –523. D’Auria, F., Eramo, A., Frogheri, M. Galassi, G.M., 1996. Accuracy quantification in SPE-1 to SPE-4 organised by IAEA. Proceedings of the Fourth International Conference on Nuclear Engineering (ICONE-4), Vol. 3, American Society of Mechanical Engineers and Japan Society of Mechanical Engineers, pp. 461 – 469. D’Auria, F., Galassi, G.M., Belsito, S., Ingegneri, M., Gatta, P., 1997a. UMAE application: Contribution to the OECD/CSNI UMS Vol. 2. DCMN Report, University of Pisa, NT 307(97) Rev. 1. D’Auria, F., Galassi, G.M., Leonardi, M. and Galetti, M.R., 1997b. Application of Fast Fourier Transform method to evaluate the accuracy of SBLOCA data base, Procedings of NURETH-8, Vol. 1, pp. 358 – 366. D’Auria, F., Mavko, B., Prošek, A., 2000. Fast fourier transform based method for quantitative assessment of code predictions of experimental data, V: Proceedings, (ANS Order, 700282), Washington, American Nuclear Society. Frogheri, M., Prošek, A., D’Auria, F., Leonardi, M., Mavko, B., Parzer, I., 1995. Application of the FFT Method to the IAEA-SPE-2 and SPE-4. IJS Report, Institut ‘Joz' ef Stefan’, IJS-DP-7101, Rev.1. Holmström, H., 1992. Quantification of code uncertainties, Proceedings of the CSNI Specialist Meeting on Transient Two-Phase Flow – Current Issues in System Thermal – hydraulics, Aix-en-Provence, France. Hrvatin, S., Prošek, A., Kljenak, I., 2000. Quantitative assessment of the BETHSY 6.2 TC test simulation, Proceedings of the Eighth International Conference on Nuclear Engineering (ICONE-8), Baltimore, MD, USA. Kljenak, I., Prošek, A., 1996. Analysis of Similarity Between Small-Break Loss-of-Coolant Accident Scenarios Based on Identification and Ranking of Basic Phenomena, Proceedings of the third Regional Meeting ‘Nuclear Energy in Central Europe’, Nuclear Society of Slovenia, pp. 285 – 292. Kunz, R.F., Kasamala, R.F., Mahaffy, J.H., Murray, C.H., 2002. On the automated assessment of nuclear reactor systems code accuracy, Nuclear Engineering and Design 211, 245 – 272. Lapendes, D.N., 1978. Dictionary of Scientific and Technical Terms, second ed. McGraw-Hill Book Company, New York, St. Louis, p. 1041. Mavko, B., Parzer, I., Petelin, S., 1995. A modeling study of the PMK-NVH integral test facility. Nucl. Technol. 105, 231 –252. 206 A. Prošek et al. / Nuclear Engineering and Design 217 (2002) 179–206 Mavko, B., Prošek, A., D’Auria, F., 1997. Determination of code accuracy in predicting small-break LOCA experiment. Nucl. Technol. 120, 1 – 19. OECD-NEA, 1989. CSNI International Standard Problem N.21-Comparison Report vols. 1 and 2ª, OECD-CSNI Report N.162. Prošek, A., Mavko, B., 1995. Comparison of RELAP5/MOD2 and RELAP5/MOD3.1 Calculations Using FFT method, Proceedings of the second Regional Meeting: ‘Nuclear Energy in Central Europe’, Nuclear Society of Slovenia, Ljubljana, pp. 533 – 540. Prošek, A., Mavko, B., 1996. Application of qualitative analysis to IAEA-SPE-4 calculations, Proceedings of the third Regional Meeting: ‘Nuclear Energy in Central Europe’, Nuclear Society of Slovenia, Ljubljana, pp. 246 – 253. Prošek, A., Mavko, B., 1997. Accuracy Quantification of RELAP5/MOD3.1 Code for BETHSY 9.1B Test, Proceedings of the Annual Meeting on Nuclear Technology’97, INFORUM Verlags-und Verwaltungsgesellschaft mbH, pp. 169– 172. Prošek, A., Parzer, I., Mavko, B., Frogheri, M., D’Auria, F., Leonardi, M., 1995. Application of the FFT Method to an IAEA-SPE-4 Experiment Simulation, Proceedings of the Annual Meeting on Nuclear Technology ‘95, INFORUM Verlags-und Verwaltungsgesellschaft mbH, Germany, pp. 115 – 118. Prošek, A., Kljenak, I., Mavko, B., 1996. Quantitative Similarity Analysis of Small-Break Loss-of-Coolant Accident Scenarios, Proceeding of the Fourth International Conference on Nuclear Engineering (ICONE-4), Vol. 3: Safety and Reliability, American Society of Mechanical Engineers and Japan Society of Mechanical Engineers, pp. 79 – 86. Purhonen, H., Kouhia, J., Holmström, H., 1994. OECD/ NEA/CSNI International Standard Problem No. 33-Comparison Report, NEA/CNSI/R(94)24. Schultz, R.R., 1992. Methodology for quantifying calculational capability of RELAP5/MOD3 code for SBLOCAs, LBLOCAs, and operational transients, presented at First Code Assessment and Maintenance Program (CAMP) Mtg., Villigen, Switzerland.

RELATED PAPERS

RELATED TOPICS

Log In

Review of quantitative accuracy assessments with fast Fourier transform based method (FFTBM)

Review of quantitative accuracy assessments with fast Fourier transform based method (FFTBM)

Related Papers

RELATED PAPERS

RELATED TOPICS