The Validity of the Ease Expert System for Inhalation Exposures

Estimation and Assessment of Substance Exposure (EASE) is a computerized expert system developed by the UK Health and Safety Executive to facilitate exposure assessments in the absence of exposure measurements. The system uses a number of rules to predict a range of likely exposures or an 'end-point' for a given work situation. The purpose of this study was to identify a number of inhalation exposure measurements covering a wide range of end-points in the EASE system to compare with the predicted exposures. Occupational exposure data sets were identified from previous research projects or from consultancy work. Available information for each set of measurements was retrieved from archive storage and reviewed to ensure that it was adequate to enable EASE (version 2) predictions to be obtained. Exposure measurements and other relevant contextual data were abstracted and entered into a computer spreadsheet. EASE predictions were then obtained for each task or job and entered into the spreadsheet. In addition, we generated a random exposure range for each data set for comparison with the EASE predictions. Finally, we produced exposure assessments for a subset of the data using a structured subjective assessment method. We were able to identify $4000 inhalation exposure measurements covering 52 different scenarios and 28 EASE end-points. The data included measurements of solvent vapours, non-fibrous dusts and fibres. In 62% of the end-points the EASE predictions were generally greater than the exposure measurements and in 30% of the end-points the EASE estimates were comparable with the measurements. The random allocation of exposure ranges was, as expected, less reliable than EASE, although there were still about one-third of the cases where the randomly generated exposure ranges generally agreed with the measurements. The structured subjective assessments undertaken by a human expert produced exposure estimates in better agreement with the measurements with about two-thirds of the end-points derived from these assessments in good agreement with the data. We argue that the inhalation exposure estimates from EASE could be improved by incorporating some of the parameters included in the structured subjective assessment methodology.


INTRODUCTION
The UK Health and Safety Executive (HSE) developed a rule-based expert system to estimate inhalation and dermal exposure to hazardous substances.The system guides the user towards an estimated range of exposure levels via a series of questions about the physical properties of the substance, the pattern of use and the controls in place.It is not explicit whether the estimated exposure levels are task-specific or 8 h time-weighted average exposures for particular jobs.However, the exposure ranges or 'end-points' were based on exposure data collected by the HSE and held in their National Exposure Database (NEDB) (Burns and Beaumont, 1989), some of which were representative of the 8 h time-weighted average exposure.
Prior to this study there were a small number of case-studies and reviews that had investigated the validity of the predictions from EASE (Vincent et al., 1996;Devillers et al., 1997;ECETOC, 1997;van Rooij and Jongeneelen, 1999).The general picture provided by this information was that EASE tended to overestimate exposure, although there was a great deal of variability in the predictions in relation to measured exposures.A more comprehensive review of the studies that shed light on the validity of EASE is provided by Creely et al. (2004).
The purpose of this study was to undertake a comprehensive test of EASE predictions using occupational inhalation exposure information obtained from past research and consultancy projects undertaken at the Institute of Occupational Medicine (IOM).While there is a facility within the EASE model to estimate dermal exposures, the validation trials reported here relate only to inhalation exposure predictions.We report a separate exercise designed to explore the validity of EASE for dermal exposure in a companion paper (Hughson and Cherrie, 2004).In addition to the comparison with historical data, we applied an alternative structured subjective approach to exposure assessment to a subset of the data (Cherrie et al., 1996, Cherrie andSchneider, 1999) along with randomly assigned exposure ranges.These data are compared with the EASE predictions and the measurement data.

METHODS
A range of occupational exposure data sets were identified as being potentially suitable for the validation tests.We included all research carried out by the IOM since 1992 where inhalation exposure measurements had been made and all short-term consultancy investigations over the previous two years where similar data was available.The choice of cut-off dates for obtaining data was dependent on the availability of measurement data to the investigators.Some of the projects had originally been funded by the HSE and the participating organizations were aware that the exposure data was to be reported to HSE.In these cases, we assumed that further permission was not necessary before using the data in this study.In all other instances, the original sponsors of the work were contacted and permission was obtained before using the data for the EASE comparisons.Data was not used where we were unable to obtain a response from the organization concerned or they refused to give their permission.
All of the exposure measurements used in this study were collected by experienced occupational hygiene scientists or technicians and analysed using appropriate laboratory techniques.In all cases there was good contextual information about the various tasks to make judgements about the method of handling, level of control and other factors that are required by the EASE model.The original survey notes and final reports relating to the data sets were retrieved from archives and scrutinized to ensure that they were complete.To ensure that descriptive information was sufficient for EASE we rejected any data sets not meeting the core requirements proposed by the European Foundation for the Improvement of Living and Working Conditions (Rajan et al., 1997).
The exposure information was summarized and entered into a computer spreadsheet, which were then grouped by substance and EASE decision criteria.Exposure estimates were obtained using EASE (version 2) for each exposure category and these were also entered into the spreadsheet.The EASE programme comprises a hierarchical series of rules that are contingent on information about the substances, processes and circumstances of use.The user is prompted with a series of questions, e.g. is the physical state of the substance being assessed a gas or vapour, a liquid or a solid?Depending on the response further questions are presented on, e.g. the volatility of the material, the likelihood of aerosol formation and the pattern of control.Ultimately the software provides a predicted rage of exposures for the given circumstances, known as an 'end-point'.
The arithmetic mean and standard deviation were calculated for each group of data for which a single EASE prediction had been made.The measured exposures were then compared with the EASE predictions and the number of measurements falling within the predicted ranges were obtained.These data were used to obtain the percentage of exposure measurements within the predicted EASE range.In addition, the numbers of measurements below and above the predicted ranges were counted and these were used to determine whether the estimate was 'good', 'too high' or 'too low'.For the purpose of this study, the estimate was considered to be 'good' if >50% of measurements fell within the predicted range.When <50% of the measurements were within the predicted range and the majority of measured exposures were within or below the predicted range, then the estimated exposure was considered to be 'too high'.The estimate was considered to be 'too low' if <50% of the exposures were within the predicted range and the majority of measurements were either within the range or above the upper limit of the predicted range.
To further evaluate the reliability of the EASE predictions we compared the measurements grouped by EASE end-point with randomly assigned exposure ranges, as suggested by Esmen (1991).He proposed that it is a necessary attribute of any exposure assessment method that it should perform better than a random allocation of an exposure end-point from a plausible range of end-points.Therefore if EASE was reliable we would expect that a greater proportion of the measured exposures would be within the EASE predicted end-point range than within the randomly assigned end-point range.
We assumed that the plausible range of end-points would comprise the EASE end-points for the state of the substance being studied, i.e. gas or vapour, liquid or aerosol.Each of the EASE end-points was allocated an integer code.A random number was then generated for each set of data, i.e. the groups of measurements for which EASE predictions were made, as described above, and the appropriate exposure end-point range was selected.The percentage of measurements within the randomly selected range was then calculated as before.
Finally, for a selection of the available measurement data the exposure was estimated by an experienced occupational hygienist using a structured subjective assessment method (Cherrie et al., 1996, Cherrie andSchneider, 1999).The method involved subdivision of the job being assessed into tasks and then for each task the exposure level was estimated.Task exposure levels were estimated separately for near-field (within 1 m of the subject) and far-field (>1 m) contributions to exposure and these were then combined.The assessment was based on a multiplicative model of exposure involving factors for intrinsic emission (e.g.dustiness), the method of handling the material, the effectiveness of local controls, the time the sources of exposure were active and the effect of general ventilation.For this study no allowance was made for the reduction of exposure by respiratory protection because the results were to be compared with exposure measurements.Taskbased estimates of exposure were then combined to produce a time-weighted average estimate of exposure for the job.
Only those data sets with 10 or more measurements were selected for the structured subjective assessment method.To avoid any possibility of bias, where exposure assessments had previously been made for a set of data by this hygienist these data were re-used.For the remaining data sets the measurements recorded in the spreadsheets and the EASE predictions were hidden and the contextual information, supplemented with descriptive information in the paper reports, was used to make the assessments.Where necessary to help clarify any ambiguities, further enquiries were made with the hygienist responsible for making the original measurements.
The exposure estimates from this process were single values representing average exposure, rather than the ranges that EASE outputs.To make the data comparable with EASE estimates we selected the end-point range closest to the estimated value and allocated this to the data set, i.e. the end-point where the difference between the mid-point of the exposure range and the estimated value was smallest.Summary statistics were calculated as before.

Identification of suitable exposure data for the validation
Research data was obtained from five studies for nine different agents.The two largest data sets were obtained from epidemiological studies of workers in the heavy clay industry (Love et al., 1994) and the Scottish quarry industry (Davies et al., 1994).These two studies contributed 1988 measurements of respirable dust and quartz.
Many workers within these industries were exposed to dusty conditions for the majority of their working shift.It is therefore reasonable to assume that the daily average exposures were equivalent to the task-related activity and it is therefore a simple matter to categorize these tasks using the EASE criteria.However, certain workers were exposed to dusty conditions infrequently or for short periods.Therefore, the measured 8 h time-weighted average exposures were not always easy to match to EASE end-points.It should also be noted that EASE does not differentiate between respirable dust and total inhalable dust, producing the same prediction in both cases.No attempt was made to evaluate exposure to respirable quartz as a component of the respirable dust since there is also no mechanism within EASE to predict exposure to component parts of mixtures.In many cases the work in the quarries and heavy clay industry was carried out outdoors, and/or inside a cab, either on a vehicle or a fixed cab adjacent to a source of dust exposure.It was difficult to categorize these jobs in terms of the EASE model, which does not explicitly include these factors.
Two studies provided inhalation exposure data for fibrous dusts.There were 79 measurements available for para-aramid fibres (Cherrie et al., 1995) and 149 asbestos fibre concentrations from a study of the effectiveness of respiratory protective equipment (Howie et al., 1996).The remaining research data came from a study to compare different samplers used for measuring exposure to vapours (Cherrie et al., 1994).These data comprised between 78 and 121 measurements for styrene, butan-2-one, toluene, 1,1,2-trichlorotrifluororethane, xylene, benzene, dichloromethane and butyl acetate (most as components in mixtures).
There were 609 measurements from 10 factories where measurements were collected to assess compliance with occupational exposure limits.The two largest data sets came from a man-made mineral fibre factory (323 measurements assessed for fibre number concentration and mass concentration) and a tunnelling operation through soil contaminated with hydrocarbons (233 measurements for benzene).The remaining eight compliance surveys contributed between three and eleven measurements of asbestos fibre concentration, respirable dust, total inhalable dust, toluene, butan-2-one or acetone exposure levels.There were a further 13 surveys where non-fibrous aerosols had been measured and 12 where vapours had been measured.These surveys contributed an additional 126 data points.The aerosols included respirable and total inhalable dust measurements for various metals and minerals, welding fume, wood dust and epoxy powder.The vapour measurements included xylene, toluene, acetone, isopropyl 127 EASE expert system for inhalation exposures Downloaded from https://academic.oup.com/annweh/article-abstract/49/2/125/146225 by guest on 17 March 2019 alcohol, ethanol, dichloromethane, 4,4 0 -methylene diphenyl diisocyanate, white spirit and mercury.
Comparison of the EASE predictions with the exposure data The data are summarized in Tables 1-3 according to the category of substance measured, i.e. respirable or inhalable dusts, vapours measured or fibrous dust.For each category, the appropriate table shows the arithmetic mean, standard deviation and the number of measurements, the EASE range, the proportion of measurements within the range and the assessment of the reliability of the EASE prediction, i.e. 'good', 'too high' or 'too low'.
Considering the non-fibrous dusts (Table 1), in seven cases we judged that the EASE prediction was 'good' with 70-100% of measurements falling within the predicted range.In six cases the estimates were 'too high' and between 27 and 0% of the measurements were inside the range predicted by EASE.In the cases where the EASE predictions were judged to be 'good' the range predicted by EASE was generally low and included zero as the lower bound.The end-point criteria for these situations mostly included low dust techniques for handling the hazardous substances (coded LDT in the table, see the Appendix for a full list of abbreviations).
For the vapour comparisons (Table 2) there were five sets of data that were considered to be in good agreement with the predictions, with 53-100% of measurements within the EASE range.There were 19 cases where the EASE predictions were 'too high' and 1 where the EASE range was 'too low'.The EASE predictions that were categorized as being in 'good' agreement with the measurements were the generally lower ranges and four of them included zero as the lower bound.For most of the data sets where the prediction was 'too high' there was no measurement within the range and in no case was the average measured exposure within the EASE range.
In four of the data sets where fibre concentrations had been measured (Table 3) the EASE predictions were judged to be 'good'.These were again generally at the lowest predicted levels.EASE has a 'zero' exposure category for fibres and for these situations the exposure predictions were categorized as 'too low'.However, if we had interpreted 'zero' as <0.1 fibres ml À1 , rather than <0.01 fibres ml À1 as we selected, then all of these situations would have been reclassified as having 'good' predictions.In the seven remaining cases the predicted ranges were classed as 'too high' (four asbestos, two MMVF and one para-aramid).

Random allocation of exposure ranges
The random allocation of exposure ranges was undertaken for all of the data sets for which EASE end-points had previously been derived.
There were 21 end-point ranges where there were no measurements within the range (40%) and a further 14 end-points where less than half of the measurements were in the range (26%).In nine cases either all, or almost all, of the measurements were within the randomly selected exposure range.
More of the randomly selected vapour end-point ranges had less than half of the measurements within them.For vapours, only 24% of the comparisons had more than half the appropriate measurements in the range whereas the corresponding figure for the aerosol measurements was 47% These proportions are very similar to those found when EASE was used to estimate exposure.This is almost certainly because there are more available ranges for vapours than for dusts and the ratio of the highest to lowest value in each range is smaller for vapours, i.e. there is a lower probability that a random selection may be within the range.For vapours the average ratio between the high and low end-point values is 2.3, while for non-fibrous dusts the corresponding value is 5.4 and for fibrous dusts 36, discounting the 'zero' ranges.
Figure 1 compares the proportion of exposure measurements inside the randomly selected range with the corresponding value derived from the exposure range allocated by EASE.The data in this graph show the 52 scenarios for 28 different end-points.The bottom right hand area on the graph shows data sets where EASE was more successful than the random allocation of exposure and the top left area where the random allocation was better than EASE.Ideally, we would have liked to have seen most of the points in the former area, although in fact there is no indication that EASE is substantially better than a random allocation of exposure range.In 39% of the end-points the EASE prediction was better than the random allocation, whereas there were 22% of end-points where the random allocation produced a better agreement with the measured exposures.In the remainder of instances the EASE and randomly allocated exposure endpoints were equally good (or poor) in the assessment of exposure.

Assessment of Exposure by a Human Expert
A structured subjective exposure assessment was completed by one of the authors (J.W.C) for the 32 surveys where there were 10 or more measurements of exposure.These data are summarized in Fig. 2, along with the corresponding data from the EASE predictions and the random allocation of exposure.
There were seven surveys where non-fibrous dusts had been measured, four where fibrous dusts had been measured and seven where vapours had been measured.In about two-thirds of these, the structured subjective assessment by the human expert contained more then half of the exposure measurements, whereas the corresponding data for the other    assessments strategies were 39% for EASE and 22% for the random allocation.From the figure it can be seen that the results from the human expert were mostly in the upper left half of the figure, indicating that he was generally more reliable than EASE or a random allocation of EASE end-points.
As we have noted earlier, all three methods of exposure assessment were poorer at assessing vapours than for dusts or fibres.

DISCUSSION
EASE was used to predict exposures for 52 workplace scenarios corresponding to 28 EASE endpoints, for non-fibrous dusts, vapours and fibres.There were almost 4000 individual exposure measurements contributing to this work.In 32 of the scenarios EASE generally over-estimated exposure when compared with the measurement data.The EASE predictions for 16 of the 52 scenarios compared favourably with the hygiene data, and 4 EASE predictions under-estimated exposures.
However, EASE does contain some level of 'expertise' and produces exposure estimates that are a little better than those obtained from a randomly chosen exposure range.The key area where EASE is successful is in predicting situations with low exposure.
The evaluations of EASE carried out prior to this investigation produced results that were similar to our own.A series of case-studies are described by Devillers et al. (1997) and Vincent et al. (1996), including paint stripping, manufacture of polyurethane foam and involved substances such as dicloromethane, chlorine and total inhalable dust.In five of their case-studies the measured exposure levels were below the EASE predictions, in three the measurements broadly agreed with the measurements and in the remaining two the measurements were higher than the predictions of EASE.Van Rooij and Jongeneelen (1999) described 61 scenarios where they had measured ranges of exposure to compare with the predictions from EASE.They found that there was an overall tendency for the predictions to overestimate exposure.
Fig. 1.Comparison of the proportion of measurements inside the EASE and randomly allocated EASE range.Note there are 17 points for which the percentage in both ranges was zero.In our study, the EASE predictions for non-fibrous dusts were judged to be good in about half of the scenarios, although in the remainder the estimates were too high.While EASE requires information about the size fraction of dust to be assessed, i.e. inhalable or respirable, the decision logic does not appear to make use of this information in the prediction.Making some allowance for this could help improve the accuracy of the predictions.In addition, there is no facility within EASE to evaluate exposure for mixtures of solids, e.g.quartz in mixed respirable dust.We feel that future development of the software should acknowledge this problem and adjust the estimates to account for components in mixtures.
The apparent failure of EASE to reliably predict exposures to fibrous dusts in three of the end-point groups arises from an artefact related to the estimated 'zero' exposure level for these situations.In practice, occupational hygiene measurements of respirable fibre exposures will generate measurements above zero, so the comparison is perhaps unfair.In our comparisons we chose to interpret <0.01 fibres ml À1 as 'zero', although if these endpoint ranges were changed to <0.1 fibres ml À1 this would probably be more meaningful and would improve the reliability of EASE for fibrous dusts.
In most cases where EASE was used to predict vapour exposures the model consistently overestimated by a large margin.The reliability of the predictions was also poorer than that for the aerosols.We noted a similar pattern when we allocated the exposure ranges at random.This pattern is probably due to another artefact, namely the larger number of end-point ranges for vapours and the smaller range of exposures for each end-point, although we cannot discount differences in the pattern of use of the materials between aerosols and vapours as the cause.The choice of exposure ranges for EASE end-points needs careful review.In some cases the limits are very wide, e.g.5-50 mg m À3 and, as we have seen, in other instances they are narrower than seems justified, e.g.'zero'.The whole concept of fixed ranges may be inappropriate since the selected range seems to bear little relation to the actual uncertainty of the predictions or variability of the exposure.The use of Monte Carlo modelling techniques could provide some way of preserving information about the uncertainty in the predictions.
The EASE model generates its exposure predictions from information about the work tasks and control measures used.The model does not take into account other exposure determinants, such as the size of the workroom, the duration of exposure, frequency of exposure periods or the shift duration.The structured subjective assessments made by one of the authors, which does include such factors, were generally better than those obtained by EASE.In two-thirds of the situations evaluated the assessments were judged to be in 'good' agreement with the exposure measurements.The comparable figure for EASE was 39 and 22% for randomly selected exposure ranges.This suggests that there is considerable scope to improve the EASE decision logic to better reflect the important exposure determinants.
These data and other investigations have shown that predictions of exposure made using EASE can be similar or higher than measured exposure.We believe that the degree of variation in the estimates is unacceptable and if EASE is to remain a credible tool for regulatory risk assessments the underlying model must be extensively revised.In our study we observed greater reliability from the human assessor using the model developed by Cherrie, Schneider and others in predicting inhalation exposure when compared with EASE.This model may provide an appropriate basis from which to develop an improved method of predicting exposures.

Fig. 2 .
Fig. 2. Comparison of the reliability of three methods of predicting exposure.

Table 1 .
Summary of the reliability of the EASE predictions for respirable and total inhalable dust measurements Downloaded from https://academic.oup.com/annweh/article-abstract/49/2/125/146225 by guest on 17 March 2019 a See abbreviations in the Appendix for an explanation of the meaning of these terms.b EASE range generally higher than the measured exposure values.

Table 2 .
Summary of the reliability of the EASE predictions for vapour measurements

Table 3 .
Summary of the reliability of the EASE predictions for fibrous dust measurements